Oct 30 20:37:18.534320 ip-10-0-135-178 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 20:37:18.929088 ip-10-0-135-178 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:37:18.929088 ip-10-0-135-178 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 20:37:18.929088 ip-10-0-135-178 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:37:18.929088 ip-10-0-135-178 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:37:18.934068 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.933952    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.934068 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934064    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.934068 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934070    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.934068 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934074    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.934068 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934079    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934083    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934088    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934092    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934096    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934100    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934104    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934108    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934111    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934115    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934130    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934134    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934140    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934147    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934151    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934156    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934160    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934164    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934168    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.934382 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934172    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934176    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934180    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934184    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934187    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934191    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934195    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934199    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934203    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934207    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934211    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934215    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934220    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934224    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934230    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934236    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934240    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934244    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934249    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934253    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.935164 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934258    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934262    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934267    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934271    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934279    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934284    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934291    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934295    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934300    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934304    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934308    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934313    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934317    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934322    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934326    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934330    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934334    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934339    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934343    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.935964 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934347    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934352    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934356    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934360    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934365    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934369    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934374    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934378    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934382    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934387    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934391    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934395    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934399    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934404    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934408    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934413    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934417    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934421    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934426    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934430    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.936799 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934435    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934439    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934444    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934448    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934452    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934458    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934462    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934466    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934470    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934474    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934479    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934483    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934487    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934493    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934497    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934502    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934507    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934511    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934516    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934520    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.937667 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934525    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934529    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934534    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.934541    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935887    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935897    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935902    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935906    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935910    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935915    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935919    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935924    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935929    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935933    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935938    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935942    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935946    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935951    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935956    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.938282 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935959    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935963    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935967    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935972    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935976    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935980    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.935985    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936007    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936012    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936017    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936021    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936025    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936029    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936034    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936038    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936043    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936047    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936051    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936055    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936059    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.938779 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936064    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936068    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936072    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936077    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936081    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936086    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936091    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936095    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936099    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936103    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936108    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936112    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936117    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936122    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936127    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936131    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936136    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936140    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936149    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.939508 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936155    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936160    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936165    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936170    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936174    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936179    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936183    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936188    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936192    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936196    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936200    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936205    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936209    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936213    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936218    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936222    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936226    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936230    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936235    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936239    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.940329 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936244    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936248    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936252    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936257    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936262    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936267    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936271    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936275    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936279    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936283    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936287    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936292    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936297    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936301    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936306    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936310    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936314    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936319    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936323    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.941215 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936327    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936332    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936340    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936345    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936350    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936355    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936359    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936364    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936368    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936372    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936376    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936381    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.936385    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937274    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937284    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937289    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937294    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937298    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937304    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937313    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.941745 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937318    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937322    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937327    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937333    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937340    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937345    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937349    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937354    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937358    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937363    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937367    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937371    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937375    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937380    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937384    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937389    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937393    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937397    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937402    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.942474 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937406    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937411    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937415    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937419    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937423    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937428    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937435    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937439    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937443    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937448    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937452    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937456    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937462    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937466    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937471    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937475    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937479    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937484    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937488    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937492    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.942943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937496    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937501    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937505    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937509    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937513    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937517    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937522    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937526    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937530    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937534    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937538    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937542    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937546    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937550    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937555    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937559    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937563    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937567    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937571    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937576    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.943466 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937580    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937584    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937588    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937592    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937599    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937603    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937607    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937613    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937617    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937621    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937625    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937629    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937634    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937638    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937644    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937648    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937652    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937656    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937660    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.944027 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937664    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937668    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937672    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937676    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937682    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937690    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937695    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937700    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937704    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937711    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937715    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937719    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937723    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937728    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937733    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937737    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937741    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937746    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937750    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.944535 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937754    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.937759    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.938965    2580 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939003    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939015    2580 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939022    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939030    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939036    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939043    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939050    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939055    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939060    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939103    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939109    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939114    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939119    2580 flags.go:64] FLAG: --cgroup-root=""
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939124    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939129    2580 flags.go:64] FLAG: --client-ca-file=""
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939134    2580 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939139    2580 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939147    2580 flags.go:64] FLAG: --cluster-domain=""
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939152    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939157    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939163    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 20:37:18.945334 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939168    2580 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939174    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939180    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939186    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939192    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939197    2580 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939202    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939207    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939212    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939217    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939224    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939229    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939234    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939239    2580 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939244    2580 flags.go:64] FLAG: --enable-server="true"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939248    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939255    2580 flags.go:64] FLAG: --event-burst="100"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939261    2580 flags.go:64] FLAG: --event-qps="50"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939266    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939271    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939276    2580 flags.go:64] FLAG: --eviction-hard=""
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939283    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939288    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939293    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939298    2580 flags.go:64] FLAG: --eviction-soft=""
Oct 30 20:37:18.946282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939303    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939310    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939315    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939320    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939325    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939330    2580 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939335    2580 flags.go:64] FLAG: --feature-gates=""
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939342    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939347    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939353    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939358    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939364    2580 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939369    2580 flags.go:64] FLAG: --help="false"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939374    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939379    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939384    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939389    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939395    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939400    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939405    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939410    2580 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939415    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939419    2580 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 20:37:18.947397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939424    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939430    2580 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939435    2580 flags.go:64] FLAG: --kube-reserved=""
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939440    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939445    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939450    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939454    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939459    2580 flags.go:64] FLAG: --lock-file=""
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939464    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939469    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939476    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939485    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939490    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939495    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939500    2580 flags.go:64] FLAG: --logging-format="text"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939505    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939510    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939514    2580 flags.go:64] FLAG: --manifest-url=""
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939519    2580 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939526    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939531    2580 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939538    2580 flags.go:64] FLAG: --max-pods="110"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939544    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939549    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939554    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 20:37:18.948009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939559    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939564    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939568    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939574    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939586    2580 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939591    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939596    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939602    2580 flags.go:64] FLAG: --pod-cidr=""
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939606    2580 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939611    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939616    2580 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939620    2580 flags.go:64] FLAG: --port="10250"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939625    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939630    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-067cdabc38b0fe16b"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939636    2580 flags.go:64] FLAG: --qos-reserved=""
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939641    2580 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939646    2580 flags.go:64] FLAG: --register-node="true"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939654    2580 flags.go:64] FLAG: --register-with-taints=""
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939660    2580 flags.go:64] FLAG: --registry-burst="10"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939665    2580 flags.go:64] FLAG: --registry-qps="5"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939669    2580 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939675    2580 flags.go:64] FLAG: --reserved-memory=""
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939681    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939686    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939691    2580 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939698    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 20:37:18.948612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939703    2580 flags.go:64] FLAG: --runonce="false"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939707    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939712    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939718    2580 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939723    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939728    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939733    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939738    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939743    2580 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939748    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939753    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939758    2580 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939762    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939768    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939772    2580 flags.go:64] FLAG: --system-cgroups=""
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939777    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939786    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939790    2580 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939795    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939802    2580 flags.go:64] FLAG: --tls-min-version=""
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939806    2580 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939811    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939816    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939822    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939828    2580 flags.go:64] FLAG: --v="2"
Oct 30 20:37:18.949385 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939835    2580 flags.go:64] FLAG: --version="false"
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939842    2580 flags.go:64] FLAG: --vmodule=""
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939848    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.939853    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940093    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940101    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940107    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940112    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940116    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940120    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940125    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940130    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940134    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940139    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940143    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940147    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940151    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940155    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940159    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940163    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.949976 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940168    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940172    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940177    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940181    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940185    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940189    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940193    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940198    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940202    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940206    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940213    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940217    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940222    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940226    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940230    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940234    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940239    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940244    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940250    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.950486 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940254    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940259    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940263    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940267    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940272    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940277    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940281    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940285    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940290    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940294    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940298    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940303    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940307    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940311    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940316    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940320    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940324    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940328    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940332    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940336    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.950943 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940340    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940344    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940348    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940354    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940359    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940364    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940368    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940372    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940377    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940381    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940385    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940391    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940396    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940402    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940409    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940414    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940420    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940424    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940430    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.951454 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940435    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940439    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940443    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940447    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940451    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940456    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940460    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940464    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940469    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940473    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940479    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940485    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940490    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940494    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940499    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940503    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940509    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940513    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940517    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.951916 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940521    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940525    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940529    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940533    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940538    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940544    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940548    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940553    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940557    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940561    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940565    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940569    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940574    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.940619    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940676    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940684    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.952381 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940689    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940694    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940698    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940703    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940707    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940711    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940716    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940720    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940725    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940729    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940733    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940738    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940742    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940748    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940752    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940756    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940761    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940765    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940769    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940773    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.952764 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940777    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940782    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940788    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940792    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940797    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940801    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940805    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940809    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940813    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940818    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940823    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940827    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940831    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940836    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940840    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940844    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940849    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940853    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940857    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.953276 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940861    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940866    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940871    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940875    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940880    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940884    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940890    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940894    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940898    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940902    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940906    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940911    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940915    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940919    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940923    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940929    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940933    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940937    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940942    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940946    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.953732 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940950    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940954    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940959    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940963    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940968    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940972    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940976    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940981    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.940986    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941012    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941017    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941021    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941025    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941029    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941033    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941038    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941042    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941048    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941056    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.954698 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941060    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941065    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941069    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941074    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941078    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941082    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941086    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941091    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941097    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941101    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941105    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941109    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941114    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941118    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941125    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941131    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941136    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941140    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941146    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.955188 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941150    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941155    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941159    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941163    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941167    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941171    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.941175    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.950519    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.950538    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.950555    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.950560    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950637    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950642    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950646    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950650    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950652    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950655    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950658    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950660    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.955910 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950663    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950665    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950668    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950671    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950673    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950676    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950678    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950681    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950684    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950686    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950690    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950693    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950701    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950704    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950707    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950710    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950712    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950715    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950718    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950720    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.956662 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950723    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950726    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950729    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950732    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950734    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950737    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950740    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950742    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950745    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950748    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950750    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950753    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950755    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950758    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950760    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950763    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950766    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950768    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950771    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950773    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.957270 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950776    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950778    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950781    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950783    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950786    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950795    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950800    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950802    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950805    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950808    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950810    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950813    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950816    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950819    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950822    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950825    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950828    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950830    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950833    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.957769 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950836    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950838    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950841    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950844    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950846    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950849    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950851    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950854    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950856    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950859    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950861    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950864    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950866    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950869    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950871    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950874    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950876    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950879    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950882    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950890    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.958263 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950892    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950895    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950897    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950900    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950903    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950905    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950909    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950911    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950914    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950917    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950919    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950923    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950927    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950930    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950933    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950936    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950939    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950941    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.958761 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.950944    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.950973    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951213    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951220    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951224    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951228    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951231    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951233    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951236    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951238    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951241    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951243    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951246    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951249    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951257    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951260    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.959201 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951263    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951267    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951270    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951273    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951276    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951279    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951282    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951285    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951287    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951290    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951293    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951295    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951298    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951300    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951303    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951306    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951308    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951311    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951313    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951316    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.959618 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951318    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951321    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951324    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951326    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951329    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951331    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951334    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951336    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951339    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951341    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951344    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951352    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951355    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951358    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951361    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951363    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951365    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951368    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951371    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.960156 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951373    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951376    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951378    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951381    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951383    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951386    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951389    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951392    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951394    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951396    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951399    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951401    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951404    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951406    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951409    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951411    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951414    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951416    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951418    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951421    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.960600 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951424    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951426    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951429    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951432    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951434    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951442    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951445    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951448    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951451    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951453    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951456    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951458    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951461    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951464    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951466    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951469    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951472    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951474    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951477    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951479    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.961090 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951482    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951484    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951487    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951489    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951492    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951494    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951497    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951499    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951501    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951504    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951506    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951509    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951511    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.951533    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951576    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:37:18.961552 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951580    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951583    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951586    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951598    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951601    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951604    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951607    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951610    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951612    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951615    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951618    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951620    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951623    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951625    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951628    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951631    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951634    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951636    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951640    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:37:18.961913 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951645    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951648    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951651    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951653    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951656    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951658    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951660    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951663    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951665    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951668    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951670    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951673    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951676    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951680    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951684    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951687    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951689    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951697    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951700    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:37:18.962461 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951703    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951705    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951708    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951710    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951713    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951716    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951718    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951721    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951726    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951729    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951731    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951734    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951736    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951738    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951741    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951743    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951746    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951748    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951750    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951753    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:37:18.962940 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951755    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951758    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951760    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951763    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951765    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951768    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951770    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951773    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951775    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951778    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951781    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951788    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951791    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951793    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951796    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951798    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951801    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951804    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951807    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951809    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:37:18.963421 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951813    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951816    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951818    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951820    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951823    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951825    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951828    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951831    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951836    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951839    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951841    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951844    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951847    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951849    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951852    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951854    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951857    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951860    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951862    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951865    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:37:18.963880 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951867    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951870    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951873    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951875    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951883    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951886    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:18.951888    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.951949    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.954088    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.955203    2580 server.go:1014] "Starting client certificate rotation"
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.955462    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.956324    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.959303    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:37:18.964366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.960451    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:37:18.971248 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.971227    2580 log.go:25] "Validated CRI v1 runtime API"
Oct 30 20:37:18.974914 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.974892    2580 log.go:25] "Validated CRI v1 image API"
Oct 30 20:37:18.975766 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.975742    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 20:37:18.978566 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.978546    2580 fs.go:135] Filesystem UUIDs: map[5b58bffc-db78-4e46-accc-3193eb4625c7:/dev/nvme0n1p4 69f45577-64fd-46ac-93fa-9c1ee9f6ab08:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 30 20:37:18.978615 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.978565    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 20:37:18.984534 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.984409    2580 manager.go:217] Machine: {Timestamp:2026-10-30 20:37:18.98277245 +0000 UTC m=+0.332473978 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099751 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec200d715fd40e77dd436ead884d3aad SystemUUID:ec200d71-5fd4-0e77-dd43-6ead884d3aad BootID:0289e18c-fa19-4ccf-ab6c-498e3a26f0db Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dd:91:29:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dd:91:29:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:f9:75:11:d6:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 20:37:18.984534 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.984529    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 20:37:18.984675 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.984663    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 20:37:18.988621 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.988594    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 20:37:18.988828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.988623    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-178.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 20:37:18.988875 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.988839    2580 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 20:37:18.988875 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.988853    2580 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 20:37:18.988937 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.988897    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:37:18.988968 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.988917    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 20:37:18.988968 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.988944    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:37:18.989719 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.989707    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 20:37:18.993453 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.993439    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 20:37:18.993613 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.993603    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 20:37:18.993649 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.993640    2580 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 20:37:18.993679 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.993658    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 20:37:18.993706 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.993680    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 20:37:18.993732 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.993708    2580 kubelet.go:405] "Adding apiserver pod source"
Oct 30 20:37:18.993732 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.993718    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 20:37:18.994295 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.994272    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:37:18.997956 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:18.997919    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 20:37:19.003722 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.003706    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:37:19.004446 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004433    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 20:37:19.004486 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004466    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 20:37:19.004628 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004611    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 20:37:19.004628 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004626    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 20:37:19.004694 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004633    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 20:37:19.004694 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004639    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 20:37:19.004694 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004653    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 20:37:19.004694 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.004659    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 20:37:19.005342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.005332    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 20:37:19.005342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.005341    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 20:37:19.005446 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.005349    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 20:37:19.005446 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.005355    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 20:37:19.005446 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.005363    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 20:37:19.005446 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.005373    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 20:37:19.006557 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.006546    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 20:37:19.006557 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.006556    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 20:37:19.008368 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.008346    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 20:37:19.008368 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.008345    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 20:37:19.010317 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.010303    2580 server.go:1263] "Started kubelet"
Oct 30 20:37:19.010447 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.010378    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 20:37:19.010447 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.010402    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 20:37:19.010566 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.010462    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 20:37:19.011245 ip-10-0-135-178 systemd[1]: Started Kubernetes Kubelet.
Oct 30 20:37:19.012501 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.012482    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 20:37:19.013182 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.013167    2580 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 20:37:19.016335 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.016318    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 20:37:19.017366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.017331    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 20:37:19.017855 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.017837    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 20:37:19.017987 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.017969    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:19.018415 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.018398    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 20:37:19.018415 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.018399    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 20:37:19.018543 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.018420    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 20:37:19.018608 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.018594    2580 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 20:37:19.018608 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.018607    2580 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 20:37:19.019132 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019108    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:19.019508 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019495    2580 factory.go:55] Registering systemd factory
Oct 30 20:37:19.019508 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019510    2580 factory.go:223] Registration of the systemd container factory successfully
Oct 30 20:37:19.019742 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019727    2580 factory.go:153] Registering CRI-O factory
Oct 30 20:37:19.019742 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019741    2580 factory.go:223] Registration of the crio container factory successfully
Oct 30 20:37:19.019878 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019791    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 20:37:19.019878 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019808    2580 factory.go:103] Registering Raw factory
Oct 30 20:37:19.019878 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.019823    2580 manager.go:1199] Started watching for new ooms in manager
Oct 30 20:37:19.020168 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.019044    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a7701f25a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.010271834 +0000 UTC m=+0.359973362,LastTimestamp:2026-10-30 20:37:19.010271834 +0000 UTC m=+0.359973362,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.020984 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.020970    2580 manager.go:319] Starting recovery of all containers
Oct 30 20:37:19.026494 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.026454    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cjphc"
Oct 30 20:37:19.028888 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.028848    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 20:37:19.033314 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.033294    2580 manager.go:324] Recovery completed
Oct 30 20:37:19.035407 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.035376    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 20:37:19.038446 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.038428    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.040407 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.040386    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.040471 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.040425    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.040471 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.040440    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.041022 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.041003    2580 cpu_manager.go:226] "Starting" policy="none"
Oct 30 20:37:19.041022 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.041020    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 20:37:19.041141 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.041040    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 20:37:19.041141 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.041006    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.041981 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.041963    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.042087 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.042017    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.042087 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.042031    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.043483 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.043468    2580 policy_none.go:50] "Start"
Oct 30 20:37:19.043557 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.043485    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 20:37:19.043557 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.043500    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 20:37:19.045582 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.045569    2580 policy_none.go:44] "Start"
Oct 30 20:37:19.046570 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.046501    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.053556 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.053484    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,LastTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.061396 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.061313    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,LastTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.066140 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.066077    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.041978924 +0000 UTC m=+0.391680452,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.072024 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.071923    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,LastTimestamp:2026-10-30 20:37:19.042024111 +0000 UTC m=+0.391725640,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.074639 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.074604    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 20:37:19.074939 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.074863    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,LastTimestamp:2026-10-30 20:37:19.042036735 +0000 UTC m=+0.391738267,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.077248 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.077230    2580 manager.go:341] "Starting Device Plugin manager"
Oct 30 20:37:19.077340 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.077264    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 20:37:19.077340 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.077273    2580 server.go:87] "Starting device plugin registration server"
Oct 30 20:37:19.077533 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.077519    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 20:37:19.077612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.077545    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 20:37:19.077681 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.077665    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 20:37:19.078073 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.077746    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 20:37:19.078073 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.077764    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 20:37:19.078273 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.078254    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 20:37:19.078316 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.078308    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:19.081625 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.081565    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a7aff1c19  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated limits on system reserved cgroup /system.slice,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.077194777 +0000 UTC m=+0.426896296,LastTimestamp:2026-10-30 20:37:19.077194777 +0000 UTC m=+0.426896296,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.087688 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.087633    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a7b270154  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.079809364 +0000 UTC m=+0.429510882,LastTimestamp:2026-10-30 20:37:19.079809364 +0000 UTC m=+0.429510882,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.155502 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.155463    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 20:37:19.157429 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.156688    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 20:37:19.157429 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.156712    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 20:37:19.157429 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.156735    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 20:37:19.157429 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.156778    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 20:37:19.177915 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.177899    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.179145 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.179082    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Oct 30 20:37:19.179271 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.179255    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.179331 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.179288    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.179331 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.179302    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.179331 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.179325    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.190204 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.190127    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.179272791 +0000 UTC m=+0.528974318,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.208093 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.208069    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.208529 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.208456    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,LastTimestamp:2026-10-30 20:37:19.179295897 +0000 UTC m=+0.528997425,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.221787 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.221714    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,LastTimestamp:2026-10-30 20:37:19.179306492 +0000 UTC m=+0.529008020,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.240784 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.240758    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Oct 30 20:37:19.257924 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.257878    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"]
Oct 30 20:37:19.258026 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.257962    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.260669 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.260655    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.260744 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.260682    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.260744 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.260692    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.262894 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.262881    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.263576 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.263559    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.263629 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.263606    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.263629 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.263613    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.263711 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.263636    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.263711 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.263651    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.264522 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.264509    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.264586 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.264533    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.264586 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.264543    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.265609 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.265594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.265695 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.265620    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.266306 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.266289    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.266392 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.266315    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.266392 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.266327    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.271419 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.271218    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.260669396 +0000 UTC m=+0.610370924,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.283299 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.283231    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,LastTimestamp:2026-10-30 20:37:19.260686952 +0000 UTC m=+0.610388481,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.287856 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.287840    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.289785 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.289769    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.315761 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.315696    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,LastTimestamp:2026-10-30 20:37:19.260695972 +0000 UTC m=+0.610397499,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.319818 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.319800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.319873 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.319826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.319873 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.319843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d0d587662eb50ad2820bb8aa4c2c1ec-config\") pod \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"3d0d587662eb50ad2820bb8aa4c2c1ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.329737 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.329668    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.263625478 +0000 UTC m=+0.613327017,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.338011 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.337933    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,LastTimestamp:2026-10-30 20:37:19.263642938 +0000 UTC m=+0.613344469,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.348260 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.348194    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,LastTimestamp:2026-10-30 20:37:19.263655154 +0000 UTC m=+0.613356681,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.359467 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.359410    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.264520094 +0000 UTC m=+0.614221621,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.369437 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.369372    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,LastTimestamp:2026-10-30 20:37:19.264538079 +0000 UTC m=+0.614239607,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.379759 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.379693    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,LastTimestamp:2026-10-30 20:37:19.264546915 +0000 UTC m=+0.614248443,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.389581 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.389512    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.26630352 +0000 UTC m=+0.616005055,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.402285 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.402221    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce25fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040431611 +0000 UTC m=+0.390133140,LastTimestamp:2026-10-30 20:37:19.266321813 +0000 UTC m=+0.616023342,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.408348 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.408333    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:19.409276 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.409259    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:19.409339 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.409291    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:19.409339 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.409308    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:19.409339 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.409340    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.413460 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.413402    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78ce5fa0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040446368 +0000 UTC m=+0.390147897,LastTimestamp:2026-10-30 20:37:19.266330715 +0000 UTC m=+0.616032243,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.418212 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.418155    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-178.us-east-2.compute.internal.18e3691a78cdc454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-178.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-178.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:19.040406612 +0000 UTC m=+0.390108141,LastTimestamp:2026-10-30 20:37:19.409276666 +0000 UTC m=+0.758978194,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:19.418370 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.418343    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.420080 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.420051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.420080 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.420079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.420226 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.420095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d0d587662eb50ad2820bb8aa4c2c1ec-config\") pod \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"3d0d587662eb50ad2820bb8aa4c2c1ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.420226 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.420154    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d0d587662eb50ad2820bb8aa4c2c1ec-config\") pod \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"3d0d587662eb50ad2820bb8aa4c2c1ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.420226 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.420154    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.420226 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.420183    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.592088 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.591987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.594838 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:19.594820    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:19.643986 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:19.643809    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Oct 30 20:37:21.260762 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.260731    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:21.261710 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.261685    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:21.261846 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.261728    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:21.261846 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.261743    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:21.261846 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.261780    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:21.280397 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.280360    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:21.470632 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.470554    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:21.574367 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.574335    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 20:37:21.662764 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.662732    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Oct 30 20:37:21.669994 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.669978    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:21.671276 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.671259    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:21.671380 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.671286    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:21.671380 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.671296    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:21.715858 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:21.715402    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0d587662eb50ad2820bb8aa4c2c1ec.slice/crio-985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81 WatchSource:0}: Error finding container 985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81: Status 404 returned error can't find the container with id 985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81
Oct 30 20:37:21.716094 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:21.716076    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457f0a554379acbff3f8f5b8e1c5a53f.slice/crio-8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c WatchSource:0}: Error finding container 8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c: Status 404 returned error can't find the container with id 8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c
Oct 30 20:37:21.721354 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:21.721336    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:37:21.731715 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.731639    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal.18e3691b18a1c2f5  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal,UID:457f0a554379acbff3f8f5b8e1c5a53f,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d017b7d199b1d54e889bb1eb6e572d8e6d94ad92610894009ee0562c2a5c0f51\",Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:21.721877237 +0000 UTC m=+1.630185491,LastTimestamp:2026-10-30 20:37:21.721877237 +0000 UTC m=+1.630185491,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:21.741108 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.741043    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal.18e3691b18b5246e  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal,UID:3d0d587662eb50ad2820bb8aa4c2c1ec,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e\",Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:21.723147374 +0000 UTC m=+1.631455613,LastTimestamp:2026-10-30 20:37:21.723147374 +0000 UTC m=+1.631455613,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:21.768812 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.768787    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 20:37:21.896417 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.896372    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Oct 30 20:37:21.931381 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:21.931349    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 20:37:22.081346 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.081264    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:22.082372 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.082349    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:22.082493 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.082413    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:22.082493 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.082431    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:22.082493 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.082476    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:22.101563 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:22.101536    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:22.471480 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.471404    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:22.604808 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.604741    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerStarted","Data":"8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c"}
Oct 30 20:37:22.606584 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:22.606557    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"3d0d587662eb50ad2820bb8aa4c2c1ec","Type":"ContainerStarted","Data":"985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81"}
Oct 30 20:37:23.190093 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.189996    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal.18e3691b6f869b79  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal,UID:457f0a554379acbff3f8f5b8e1c5a53f,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d017b7d199b1d54e889bb1eb6e572d8e6d94ad92610894009ee0562c2a5c0f51\" in 1.457s (1.457s including waiting). Image size: 272339155 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:23.179715449 +0000 UTC m=+3.088023688,LastTimestamp:2026-10-30 20:37:23.179715449 +0000 UTC m=+3.088023688,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:23.287189 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.287094    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal.18e3691b755928e0  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal,UID:3d0d587662eb50ad2820bb8aa4c2c1ec,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e\" in 1.554s (1.554s including waiting). Image size: 294084393 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:23.277400288 +0000 UTC m=+3.185708545,LastTimestamp:2026-10-30 20:37:23.277400288 +0000 UTC m=+3.185708545,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:23.337865 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.337779    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal.18e3691b785e91aa  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal,UID:3d0d587662eb50ad2820bb8aa4c2c1ec,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Container created,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:23.328086442 +0000 UTC m=+3.236394697,LastTimestamp:2026-10-30 20:37:23.328086442 +0000 UTC m=+3.236394697,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:23.350446 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.350341    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal.18e3691b78d7f0df  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal,UID:3d0d587662eb50ad2820bb8aa4c2c1ec,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Container started,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:23.336040671 +0000 UTC m=+3.244348935,LastTimestamp:2026-10-30 20:37:23.336040671 +0000 UTC m=+3.244348935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:23.396921 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.396821    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal.18e3691b7bd9ddfc  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal,UID:457f0a554379acbff3f8f5b8e1c5a53f,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Container created,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:23.386498556 +0000 UTC m=+3.294806811,LastTimestamp:2026-10-30 20:37:23.386498556 +0000 UTC m=+3.294806811,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:23.404850 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.404764    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal.18e3691b7c63c99c  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal,UID:457f0a554379acbff3f8f5b8e1c5a53f,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Container started,Source:EventSource{Component:kubelet,Host:ip-10-0-135-178.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 20:37:23.395537308 +0000 UTC m=+3.303845570,LastTimestamp:2026-10-30 20:37:23.395537308 +0000 UTC m=+3.303845570,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-178.us-east-2.compute.internal,}"
Oct 30 20:37:23.469190 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.469108    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:23.505655 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.505627    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Oct 30 20:37:23.611518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.611481    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"3d0d587662eb50ad2820bb8aa4c2c1ec","Type":"ContainerStarted","Data":"9cd6f1b6355cc0ef5dc6b091f8b2ed220b73dfb9de6db24cb55082dc14212439"}
Oct 30 20:37:23.611518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.611506    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:23.612861 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.612837    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerStarted","Data":"10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246"}
Oct 30 20:37:23.612953 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.612924    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:23.615347 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.615329    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:23.615451 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.615362    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:23.615451 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.615373    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:23.615451 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.615330    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:23.615451 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.615432    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:23.615451 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.615445    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:23.615629 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.615533    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:23.615629 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.615583    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:23.668217 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.668186    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Oct 30 20:37:23.702517 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.702481    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:23.703536 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.703518    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:23.703618 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.703550    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:23.703618 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.703563    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:23.703618 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.703589    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:23.721993 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.721923    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:23.864628 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.864595    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:23.865707 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.865689    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:23.865828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.865720    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:23.865828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:23.865734    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:23.964616 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:23.964584    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 20:37:24.150300 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:24.150231    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 20:37:24.473111 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.473035    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:24.613970 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.613946    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:24.614322 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.613952    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:24.614876 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.614858    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:24.614965 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.614880    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:24.614965 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.614887    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:24.614965 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.614898    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:24.614965 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.614900    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:24.614965 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:24.614910    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:24.615144 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:24.615091    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:24.615219 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:24.615206    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:24.749480 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:24.749400    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 20:37:25.472845 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:25.472814    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:26.470144 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:26.470116    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:26.715601 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:26.715562    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Oct 30 20:37:26.922742 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:26.922655    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:26.924048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:26.924029    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:26.924175 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:26.924068    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:26.924175 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:26.924083    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:26.924175 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:26.924118    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:26.944284 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:26.944248    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-178.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:27.469106 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:27.469080    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:27.863486 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:27.863410    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 20:37:28.469918 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:28.469886    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:28.483045 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:28.483023    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:28.483941 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:28.483925    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:28.484015 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:28.483954    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:28.484015 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:28.483965    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:29.049698 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:29.049661    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 20:37:29.474251 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:29.474171    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-178.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 20:37:29.794261 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:29.794180    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cjphc"
Oct 30 20:37:30.063362 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.063289    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:37:30.125366 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.125332    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:37:30.396217 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.396131    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 20:37:30.396337 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.396292    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 20:37:30.396337 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.396309    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 20:37:30.442697 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.442674    2580 apiserver.go:52] "Watching apiserver"
Oct 30 20:37:30.453850 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.453830    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 20:37:30.453964 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.453875    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=[]
Oct 30 20:37:30.460232 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.460215    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 20:37:30.466861 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.466839    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:30.483552 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.483527    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:30.519892 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:30.519861    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:30.536986 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.536965    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:30.790855 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.790782    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:30.790855 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:30.790809    2580 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:30.797921 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.797886    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 20:32:29 +0000 UTC" deadline="2028-06-06 11:57:26.000166398 +0000 UTC"
Oct 30 20:37:30.797921 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.797915    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14031h19m55.2022543s"
Oct 30 20:37:30.825777 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.825752    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:30.839645 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.839624    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:30.893907 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:30.893885    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:31.166877 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:31.166803    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:31.166877 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:31.166828    2580 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:31.424180 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:31.424100    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:31.438049 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:31.438027    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:31.494252 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:31.494228    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:31.758449 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:31.758374    2580 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:31.758449 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:31.758412    2580 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-135-178.us-east-2.compute.internal" not found
Oct 30 20:37:33.126672 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.126639    2580 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:33.344659 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.344632    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.345610 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.345596    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.345673 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.345628    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.345673 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.345638    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.345673 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.345666    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:33.352751 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.352733    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:33.352849 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.352760    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-178.us-east-2.compute.internal\": node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:33.367161 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.367141    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:33.376430 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.376409    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-29897"]
Oct 30 20:37:33.376506 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.376484    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.377317 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.377272    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.377317 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.377304    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.377317 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.377316    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.380329 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.380315    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-bsg94"]
Oct 30 20:37:33.380396 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.380376    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.380466 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.380450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.380508 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.380489    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.381122 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.381105    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.381223 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.381138    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.381223 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.381153    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.381326 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.381222    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.381326 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.381247    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.381326 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.381262    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.385398 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.385372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.385486 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.385415    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.386117 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.386101    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.386197 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.386131    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.386197 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.386147    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.392665 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.392649    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2hx48\""
Oct 30 20:37:33.393193 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.393177    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 20:37:33.393273 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.393177    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 20:37:33.393273 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.393242    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 20:37:33.394796 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.394780    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 20:37:33.400121 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.400104    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"]
Oct 30 20:37:33.400219 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.400210    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.403146 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.403131    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.403241 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.403164    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.403241 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.403180    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.406362 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.406346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.406467 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.406380    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.408089 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.408070    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.408173 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.408103    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.408173 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.408117    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.409246 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.409228    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-825zz"]
Oct 30 20:37:33.409459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.409447    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.410317 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.410302    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.410420 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.410332    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.410420 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.410346    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.413535 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.413520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.413634 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.413556    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.414293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.414279    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.414355 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.414309    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.414355 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.414326    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.467772 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.467737    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:33.499073 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.499045    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 20:37:33.499210 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.499194    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 20:37:33.500044 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500028    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 20:37:33.500096 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500082    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x97dd\""
Oct 30 20:37:33.500188 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500169    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 20:37:33.500188 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500186    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 20:37:33.500409 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500264    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 20:37:33.500409 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500322    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bcgjk\""
Oct 30 20:37:33.500537 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500440    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 20:37:33.500678 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500657    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 20:37:33.500735 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500665    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 20:37:33.500779 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500738    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-7lq4m\""
Oct 30 20:37:33.500878 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.500857    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-f7tx2"]
Oct 30 20:37:33.501032 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.501020    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.501094 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.501041    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 20:37:33.501980 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.501965    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.502048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.501994    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.502048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.502008    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.504689 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.504675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.504731 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.504701    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.505464 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.505448    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.505528 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.505476    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.505528 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.505488    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.509038 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.509024    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-p52s5"]
Oct 30 20:37:33.509185 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.509175    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.509949 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.509931    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.510024 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.509964    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.510515 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.510251    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.513813 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.513795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:33.513909 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.513826    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.514625 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.514608    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.514710 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.514640    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.514710 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.514655    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.514802 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.514728    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:33.524397 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.524474 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-sys-fs\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.524474 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-systemd-units\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524532 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-var-lib-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524565 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-log-socket\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524596 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524624 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-config\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524652 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk4kc\" (UniqueName: \"kubernetes.io/projected/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-kube-api-access-jk4kc\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.524652 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-node-log\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524719 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-registration-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.524719 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-device-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.524719 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-netd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524822 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/260a13a6-e92d-4243-991e-ba4639127adc-ovn-node-metrics-cert\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524822 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-konnectivity-ca\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.524822 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-netns\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524951 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524951 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-systemd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524951 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.524951 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-env-overrides\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.525112 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-slash\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.525112 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.524984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-etc-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.525112 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-bin\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.525112 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-script-lib\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.525112 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zbf2\" (UniqueName: \"kubernetes.io/projected/260a13a6-e92d-4243-991e-ba4639127adc-kube-api-access-8zbf2\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.525112 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-host\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.525312 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-agent-certs\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.525312 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.525312 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-kubelet\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.525312 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-serviceca\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.525312 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-socket-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.525312 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdt9f\" (UniqueName: \"kubernetes.io/projected/a8ff15b4-7b0f-4e99-883c-05938faa37df-kube-api-access-gdt9f\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.525312 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.525279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-ovn\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.568344 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.568308    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:33.592946 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.592923    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-6lk59\""
Oct 30 20:37:33.593064 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.593047    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 20:37:33.593064 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.593056    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 20:37:33.593186 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.593121    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 20:37:33.593286 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.593273    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 20:37:33.597166 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.597150    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-52cfv"]
Oct 30 20:37:33.597299 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.597289    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.598253 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.598237    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.598337 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.598266    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.598337 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.598276    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.600980 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.600967    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.601024 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.600995    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.601759 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.601744    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.601828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.601770    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.601828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.601780    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.606776 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.606756    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-2rrmv"]
Oct 30 20:37:33.606901 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.606891    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.607660 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.607638    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.607752 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.607667    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.607752 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.607681    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.610218 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.610202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.610302 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.610227    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.611008 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.610990    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.611070 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.611019    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.611070 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.611028    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.626060 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-serviceca\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.626151 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-socket-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.626151 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdt9f\" (UniqueName: \"kubernetes.io/projected/a8ff15b4-7b0f-4e99-883c-05938faa37df-kube-api-access-gdt9f\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.626151 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-ovn\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626151 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-system-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.626151 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-multus\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.626151 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.626343 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626192    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-ovn\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626343 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626227    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-socket-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.626343 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626241    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.626343 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626247    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.626343 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-sys-fs\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.626343 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-systemd-units\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626343 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-var-lib-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626357    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-systemd-units\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626365    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-sys-fs\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626367    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-var-lib-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-log-socket\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626465    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-log-socket\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-config\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626492    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-socket-dir-parent\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk4kc\" (UniqueName: \"kubernetes.io/projected/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-kube-api-access-jk4kc\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-node-log\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-bin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-kubelet\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-hostroot\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626622    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-conf-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626668    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-node-log\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.626699 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-daemon-config\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-registration-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-device-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-netd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626762    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-registration-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/260a13a6-e92d-4243-991e-ba4639127adc-ovn-node-metrics-cert\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626808    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-netd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-etc-kubernetes\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626792    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-device-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-konnectivity-ca\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626853    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-netns\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626906    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-serviceca\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcj5d\" (UniqueName: \"kubernetes.io/projected/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-kube-api-access-pcj5d\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls8xd\" (UniqueName: \"kubernetes.io/projected/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-kube-api-access-ls8xd\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-systemd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.626980    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627022    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-netns\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627052    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-config\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627054    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627083    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-systemd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-env-overrides\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-slash\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-etc-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-bin\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-script-lib\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627232    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-slash\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627246    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-etc-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zbf2\" (UniqueName: \"kubernetes.io/projected/260a13a6-e92d-4243-991e-ba4639127adc-kube-api-access-8zbf2\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627277    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-bin\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627295    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627307    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-k8s-cni-cncf-io\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627323    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627338    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.628643 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627341    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-netns\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627352    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-konnectivity-ca\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-host\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-agent-certs\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627357    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627459    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627479    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-env-overrides\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627588    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-host\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627642    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-kubelet\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627643    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627758    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627774    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627779    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-script-lib\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cnibin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627854    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-kubelet\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627889    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cni-binary-copy\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.629162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.627954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-multus-certs\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.629737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.628103    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.629737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.628130    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.629737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.628145    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.629737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.628144    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 20:37:33.629737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.628467    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.629737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.628491    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.629737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.628501    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.631714 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.631695    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/260a13a6-e92d-4243-991e-ba4639127adc-ovn-node-metrics-cert\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.631795 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.631723    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-agent-certs\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.636224 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.636197    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zbf2\" (UniqueName: \"kubernetes.io/projected/260a13a6-e92d-4243-991e-ba4639127adc-kube-api-access-8zbf2\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.636308 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.636296    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk4kc\" (UniqueName: \"kubernetes.io/projected/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-kube-api-access-jk4kc\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.636801 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.636785    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdt9f\" (UniqueName: \"kubernetes.io/projected/a8ff15b4-7b0f-4e99-883c-05938faa37df-kube-api-access-gdt9f\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.668855 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.668823    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:33.693908 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.693883    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-chh5h\""
Oct 30 20:37:33.694056 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.694043    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:37:33.694056 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.694038    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-82bc7\""
Oct 30 20:37:33.694172 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.694087    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 20:37:33.694172 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.694114    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 20:37:33.694172 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.694156    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 20:37:33.694280 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.694155    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:37:33.695199 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.695184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-29897"
Oct 30 20:37:33.699063 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.699049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:33.708553 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:33.708352    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3db454_59fb_46b2_90e9_3b7f4c11d5bb.slice/crio-07a2407b256803d7f2dad3b166aa086262eb72ab522faa08e1c2e28a9aef9466 WatchSource:0}: Error finding container 07a2407b256803d7f2dad3b166aa086262eb72ab522faa08e1c2e28a9aef9466: Status 404 returned error can't find the container with id 07a2407b256803d7f2dad3b166aa086262eb72ab522faa08e1c2e28a9aef9466
Oct 30 20:37:33.713053 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.713032    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-lhg4p"]
Oct 30 20:37:33.713193 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.713183    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.713976 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.713960    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.714051 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.713991    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.714051 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.714006    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.717753 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.717736    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-g9hwf"]
Oct 30 20:37:33.717854 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.717841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:33.717913 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.717870    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.717964 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.717938    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.719152 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.719137    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.719224 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.719166    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.719224 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.719181    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.719324 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.719227    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.719324 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.719250    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.719324 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.719259    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.719324 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.719302    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:33.721661 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.721647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.721706 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.721670    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.722375 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.722363    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.722428 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.722406    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.722428 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.722418    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.724622 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.724607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:37:33.728234 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-k8s-cni-cncf-io\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728325 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00295160-ccc3-481f-ac98-7b9e15aa7c25-host-slash\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.728325 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728325 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cnibin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728325 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cni-binary-copy\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728325 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728319    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-k8s-cni-cncf-io\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-lib-modules\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-system-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728415    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728415    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cnibin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-sys\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-socket-dir-parent\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728516    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-system-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-bin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-systemd\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-host\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.728591 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-conf\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-tuned\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-tmp\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-netns\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-multus-certs\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f65gx\" (UniqueName: \"kubernetes.io/projected/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-kube-api-access-f65gx\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-multus\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5xp2\" (UniqueName: \"kubernetes.io/projected/00295160-ccc3-481f-ac98-7b9e15aa7c25-kube-api-access-z5xp2\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-kubelet\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-conf-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-hostroot\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728780    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cni-binary-copy\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-daemon-config\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/00295160-ccc3-481f-ac98-7b9e15aa7c25-iptables-alerter-script\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728840    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728891    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-bin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.728963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysconfig\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.729204 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729019    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-netns\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729048    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-conf-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729088    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-hostroot\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729103    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-socket-dir-parent\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729238    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-multus-certs\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729373    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-daemon-config\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729564    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-kubelet\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729604    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-multus\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-etc-kubernetes\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-run\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729716    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-etc-kubernetes\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcj5d\" (UniqueName: \"kubernetes.io/projected/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-kube-api-access-pcj5d\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls8xd\" (UniqueName: \"kubernetes.io/projected/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-kube-api-access-ls8xd\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-var-lib-kubelet\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-modprobe-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-kubernetes\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729830    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729848    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729945    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729968    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.729984    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.730048 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.730054    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:33.731040 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.730118    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs podName:d5a8e9f5-484c-4848-a012-21b8ec6ecb32 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:34.230091615 +0000 UTC m=+14.138399870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs") pod "network-metrics-daemon-p52s5" (UID: "d5a8e9f5-484c-4848-a012-21b8ec6ecb32") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:33.731040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.730620    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.731040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.730628    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.731040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.730650    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.731040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.730652    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.731040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.730665    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.731040 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.730680    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.731530 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.731515    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tck7z"]
Oct 30 20:37:33.731665 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.731655    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.732302 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.732284    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.732407 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.732309    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.732407 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.732319    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.733659 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:33.733637    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ff15b4_7b0f_4e99_883c_05938faa37df.slice/crio-26cc749a619792700ce36c27c8b81337558ca7170bbe8f913d63283a549027c5 WatchSource:0}: Error finding container 26cc749a619792700ce36c27c8b81337558ca7170bbe8f913d63283a549027c5: Status 404 returned error can't find the container with id 26cc749a619792700ce36c27c8b81337558ca7170bbe8f913d63283a549027c5
Oct 30 20:37:33.737568 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.737547    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-qhl7c"]
Oct 30 20:37:33.738012 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.737989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.738101 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.738037    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.738435 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.738417    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.739327 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.739311    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.739426 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.739333    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.739426 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.739344    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.739511 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.739436    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:33.739554 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.739505    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.739554 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.739531    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.739554 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.739543    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.741991 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.741976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.742075 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.742002    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.742133 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.742099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:33.742637 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.742625    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.742704 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.742647    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.742704 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.742659    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.745239 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.745218    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcj5d\" (UniqueName: \"kubernetes.io/projected/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-kube-api-access-pcj5d\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.747875 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.747802    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls8xd\" (UniqueName: \"kubernetes.io/projected/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-kube-api-access-ls8xd\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:33.750152 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:33.750130    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260a13a6_e92d_4243_991e_ba4639127adc.slice/crio-133e69e6bd70139e6ef630e17fef5fb3a28a6eba0d54d3b3e8e5ebe048138a96 WatchSource:0}: Error finding container 133e69e6bd70139e6ef630e17fef5fb3a28a6eba0d54d3b3e8e5ebe048138a96: Status 404 returned error can't find the container with id 133e69e6bd70139e6ef630e17fef5fb3a28a6eba0d54d3b3e8e5ebe048138a96
Oct 30 20:37:33.769686 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.769668    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:33.793015 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.792988    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l4rxj\""
Oct 30 20:37:33.793015 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.793003    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fxgqk\""
Oct 30 20:37:33.793434 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.793422    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 20:37:33.793517 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.793503    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 20:37:33.793569 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.793555    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 20:37:33.793600 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.793563    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 20:37:33.816821 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.816801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f7tx2"
Oct 30 20:37:33.825221 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:33.825196    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d61b27_34af_40d5_8ab0_b06c28aa86e4.slice/crio-16e6f45a96b4924ff58c8a8fded165b6fb4071af4edbb69e9a7f8d19a707bf2a WatchSource:0}: Error finding container 16e6f45a96b4924ff58c8a8fded165b6fb4071af4edbb69e9a7f8d19a707bf2a: Status 404 returned error can't find the container with id 16e6f45a96b4924ff58c8a8fded165b6fb4071af4edbb69e9a7f8d19a707bf2a
Oct 30 20:37:33.830626 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-systemd\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.830726 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.830726 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-host\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.830726 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rl4z\" (UniqueName: \"kubernetes.io/projected/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-kube-api-access-5rl4z\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.830852 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-kubelet-config\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.830852 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:33.830852 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830819    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-systemd\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.830852 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830846    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-host\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.830977 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kww9\" (UniqueName: \"kubernetes.io/projected/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-kube-api-access-4kww9\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.830977 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-conf\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.830977 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-tuned\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.830977 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-tmp\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.830977 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830942    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.831149 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830980    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.831149 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.830941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.831149 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f65gx\" (UniqueName: \"kubernetes.io/projected/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-kube-api-access-f65gx\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831149 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831011    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-conf\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831149 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.831149 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831113    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.831314 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xp2\" (UniqueName: \"kubernetes.io/projected/00295160-ccc3-481f-ac98-7b9e15aa7c25-kube-api-access-z5xp2\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.831314 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-tmp-dir\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.831314 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cnibin\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.831314 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831265    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/00295160-ccc3-481f-ac98-7b9e15aa7c25-iptables-alerter-script\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysconfig\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-run\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-hosts-file\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831444    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysconfig\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-dbus\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-var-lib-kubelet\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831518 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831501    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-run\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831587    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-var-lib-kubelet\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-modprobe-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-kubernetes\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00295160-ccc3-481f-ac98-7b9e15aa7c25-host-slash\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831663    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-kubernetes\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-system-cni-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-lib-modules\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831706    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00295160-ccc3-481f-ac98-7b9e15aa7c25-host-slash\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831738    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-modprobe-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831773    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-sys\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831795    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.831828 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831805    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831856    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-sys\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831856    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831863    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-lib-modules\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831945    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831969    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.831983    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.832093    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.832115    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.832125    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.832274    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.832307    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.832496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.832322    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.832869 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.832783    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/00295160-ccc3-481f-ac98-7b9e15aa7c25-iptables-alerter-script\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.834451 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.834435    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-tmp\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.834536 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.834521    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-tuned\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.841868 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.841843    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f65gx\" (UniqueName: \"kubernetes.io/projected/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-kube-api-access-f65gx\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.842332 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.842315    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xp2\" (UniqueName: \"kubernetes.io/projected/00295160-ccc3-481f-ac98-7b9e15aa7c25-kube-api-access-z5xp2\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.870399 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.870355    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:33.912935 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.912832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:37:33.919904 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.919886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:37:33.922331 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:33.922311    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00295160_ccc3_481f_ac98_7b9e15aa7c25.slice/crio-82f947d66fcee3374b788f0d89f84f777eca2fef04a91f5d489cbe4ecb999117 WatchSource:0}: Error finding container 82f947d66fcee3374b788f0d89f84f777eca2fef04a91f5d489cbe4ecb999117: Status 404 returned error can't find the container with id 82f947d66fcee3374b788f0d89f84f777eca2fef04a91f5d489cbe4ecb999117
Oct 30 20:37:33.927968 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:33.927946    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1a0ec9_f2ec_4e98_b8a0_a6f49a9e52d0.slice/crio-e813eb0da64fa852b64d5c1dbe69c362150d07cacd4346f86b7d9471790c61a1 WatchSource:0}: Error finding container e813eb0da64fa852b64d5c1dbe69c362150d07cacd4346f86b7d9471790c61a1: Status 404 returned error can't find the container with id e813eb0da64fa852b64d5c1dbe69c362150d07cacd4346f86b7d9471790c61a1
Oct 30 20:37:33.932049 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.932147 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rl4z\" (UniqueName: \"kubernetes.io/projected/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-kube-api-access-5rl4z\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.932147 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-kubelet-config\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.932147 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932149    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-kubelet-config\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932159    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kww9\" (UniqueName: \"kubernetes.io/projected/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-kube-api-access-4kww9\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932177    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932221    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932269    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-tmp-dir\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.932293 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cnibin\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-hosts-file\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-dbus\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932399    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-hosts-file\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932408    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cnibin\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-system-cni-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932531    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-system-cni-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932539    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-dbus\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932660    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-tmp-dir\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.932755 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932682    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932825    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932874    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932887    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.932984    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933019    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933034    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933121    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933144    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933155    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933161    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933178    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.933227 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933188    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.933626 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933176    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:33.933626 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933325    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:33.933626 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.933340    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:33.933626 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.933419    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:33.933626 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.933461    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret podName:754d8def-099b-4fdb-859c-0f318bf5fd38 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:34.433444528 +0000 UTC m=+14.341752766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret") pod "global-pull-secret-syncer-tck7z" (UID: "754d8def-099b-4fdb-859c-0f318bf5fd38") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:33.939204 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.939190    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:37:33.939263 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.939212    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:37:33.939263 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.939220    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5jggs for pod openshift-network-diagnostics/network-check-target-lhg4p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:33.939263 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.939255    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs podName:0ca03c7b-5428-4123-980b-9bdbff49850f nodeName:}" failed. No retries permitted until 2026-10-30 20:37:34.439244013 +0000 UTC m=+14.347552265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5jggs" (UniqueName: "kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs") pod "network-check-target-lhg4p" (UID: "0ca03c7b-5428-4123-980b-9bdbff49850f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:33.941165 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.941143    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rl4z\" (UniqueName: \"kubernetes.io/projected/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-kube-api-access-5rl4z\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:33.941234 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:33.941169    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kww9\" (UniqueName: \"kubernetes.io/projected/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-kube-api-access-4kww9\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:33.970959 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:33.970927    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.035415 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.035371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:37:34.045024 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:34.044999    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7b0506_8e4a_43bd_b7a6_e58e9879d893.slice/crio-fc0015f062976097d9cef8376e1aa48796a3346add7ae17c284845f69a59f688 WatchSource:0}: Error finding container fc0015f062976097d9cef8376e1aa48796a3346add7ae17c284845f69a59f688: Status 404 returned error can't find the container with id fc0015f062976097d9cef8376e1aa48796a3346add7ae17c284845f69a59f688
Oct 30 20:37:34.067521 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.067496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:37:34.071003 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.070988    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.076874 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:34.076846    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b8f9f1_3e33_40b5_bcc2_743909a0295f.slice/crio-1b17de172d54ac318797511723de1c03cb3979b7502b42d8e3e67a8e08ca3df4 WatchSource:0}: Error finding container 1b17de172d54ac318797511723de1c03cb3979b7502b42d8e3e67a8e08ca3df4: Status 404 returned error can't find the container with id 1b17de172d54ac318797511723de1c03cb3979b7502b42d8e3e67a8e08ca3df4
Oct 30 20:37:34.171168 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.171062    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.234740 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.234699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:34.234740 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.234852    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:34.236775 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.236125    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:34.236775 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.236155    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:34.236775 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.236171    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:34.236775 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.236277    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:34.236775 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.236321    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs podName:d5a8e9f5-484c-4848-a012-21b8ec6ecb32 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:35.236303325 +0000 UTC m=+15.144611578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs") pod "network-metrics-daemon-p52s5" (UID: "d5a8e9f5-484c-4848-a012-21b8ec6ecb32") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:34.271349 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.271314    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.372294 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.372248    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.437492 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.437043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:34.437492 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.437187    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:34.440182 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.439440    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:34.440182 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.439470    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:34.440182 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.439485    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:34.440182 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.439613    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:34.440182 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.439663    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret podName:754d8def-099b-4fdb-859c-0f318bf5fd38 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:35.439645426 +0000 UTC m=+15.347953668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret") pod "global-pull-secret-syncer-tck7z" (UID: "754d8def-099b-4fdb-859c-0f318bf5fd38") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:34.473065 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.473022    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.537916 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.537881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:34.538142 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.538123    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:34.542162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.541051    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:34.542162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.541109    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:34.542162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.541128    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:34.542162 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.541349    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:37:34.542162 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.541371    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:37:34.542162 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.541405    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5jggs for pod openshift-network-diagnostics/network-check-target-lhg4p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:34.542162 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.541456    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs podName:0ca03c7b-5428-4123-980b-9bdbff49850f nodeName:}" failed. No retries permitted until 2026-10-30 20:37:35.541438713 +0000 UTC m=+15.449746967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5jggs" (UniqueName: "kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs") pod "network-check-target-lhg4p" (UID: "0ca03c7b-5428-4123-980b-9bdbff49850f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:34.574131 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.574072    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.640987 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.640947    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerStarted","Data":"1b17de172d54ac318797511723de1c03cb3979b7502b42d8e3e67a8e08ca3df4"}
Oct 30 20:37:34.648432 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.648380    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g9hwf" event={"ID":"9c7b0506-8e4a-43bd-b7a6-e58e9879d893","Type":"ContainerStarted","Data":"fc0015f062976097d9cef8376e1aa48796a3346add7ae17c284845f69a59f688"}
Oct 30 20:37:34.654763 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.654704    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2rrmv" event={"ID":"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0","Type":"ContainerStarted","Data":"e813eb0da64fa852b64d5c1dbe69c362150d07cacd4346f86b7d9471790c61a1"}
Oct 30 20:37:34.661540 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.661508    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"133e69e6bd70139e6ef630e17fef5fb3a28a6eba0d54d3b3e8e5ebe048138a96"}
Oct 30 20:37:34.674112 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.674014    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52cfv" event={"ID":"00295160-ccc3-481f-ac98-7b9e15aa7c25","Type":"ContainerStarted","Data":"82f947d66fcee3374b788f0d89f84f777eca2fef04a91f5d489cbe4ecb999117"}
Oct 30 20:37:34.674306 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.674277    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.683710 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.683677    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f7tx2" event={"ID":"f0d61b27-34af-40d5-8ab0-b06c28aa86e4","Type":"ContainerStarted","Data":"16e6f45a96b4924ff58c8a8fded165b6fb4071af4edbb69e9a7f8d19a707bf2a"}
Oct 30 20:37:34.692463 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.691627    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt" event={"ID":"a8ff15b4-7b0f-4e99-883c-05938faa37df","Type":"ContainerStarted","Data":"26cc749a619792700ce36c27c8b81337558ca7170bbe8f913d63283a549027c5"}
Oct 30 20:37:34.696296 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.696261    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bsg94" event={"ID":"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb","Type":"ContainerStarted","Data":"07a2407b256803d7f2dad3b166aa086262eb72ab522faa08e1c2e28a9aef9466"}
Oct 30 20:37:34.700494 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.700465    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-29897" event={"ID":"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087","Type":"ContainerStarted","Data":"cde8e180e9ed2e6e35293fbf2716b4ab30b1935b98447327c5b8135811ee0a52"}
Oct 30 20:37:34.710809 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.710780    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal_457f0a554379acbff3f8f5b8e1c5a53f/setup/0.log"
Oct 30 20:37:34.710942 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.710852    2580 generic.go:356] "Generic (PLEG): container finished" podID="457f0a554379acbff3f8f5b8e1c5a53f" containerID="10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246" exitCode=1
Oct 30 20:37:34.710942 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.710885    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerDied","Data":"10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246"}
Oct 30 20:37:34.711106 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.711025    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:34.713307 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.713283    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:34.713437 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.713322    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:34.713437 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:34.713337    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:34.713655 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.713636    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:34.775228 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.775027    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.879482 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.879449    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:34.980547 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:34.980049    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.080787 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.080751    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.181718 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.181684    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.244693 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.244612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:35.244897 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.244860    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:35.251609 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.251328    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:35.251609 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.251363    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:35.251609 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.251378    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:35.251609 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.251523    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:35.251609 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.251582    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs podName:d5a8e9f5-484c-4848-a012-21b8ec6ecb32 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:37.251561736 +0000 UTC m=+17.159869991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs") pod "network-metrics-daemon-p52s5" (UID: "d5a8e9f5-484c-4848-a012-21b8ec6ecb32") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:35.282277 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.282232    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.383298 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.383266    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.446289 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.446251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:35.446493 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.446459    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:35.459109 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.459059    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:35.459109 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.459099    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:35.459109 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.459113    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:35.459360 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.459274    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:35.459360 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.459335    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret podName:754d8def-099b-4fdb-859c-0f318bf5fd38 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:37.459312946 +0000 UTC m=+17.367621207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret") pod "global-pull-secret-syncer-tck7z" (UID: "754d8def-099b-4fdb-859c-0f318bf5fd38") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:35.484419 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.484369    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.547259 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.547179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:35.547512 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.547338    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:35.550222 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.550199    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:35.550344 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.550240    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:35.550344 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.550255    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:35.550477 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.550430    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:37:35.550477 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.550446    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:37:35.550477 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.550459    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5jggs for pod openshift-network-diagnostics/network-check-target-lhg4p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:35.550629 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.550512    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs podName:0ca03c7b-5428-4123-980b-9bdbff49850f nodeName:}" failed. No retries permitted until 2026-10-30 20:37:37.550493158 +0000 UTC m=+17.458801411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5jggs" (UniqueName: "kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs") pod "network-check-target-lhg4p" (UID: "0ca03c7b-5428-4123-980b-9bdbff49850f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:35.585592 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.585521    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.599896 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.599376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:35.599896 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.599438    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:35.599896 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.599672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:35.599896 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.599698    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:35.599896 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.599737    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:35.599896 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.599752    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601059    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601094    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601107    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.601304    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601469    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601497    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601513    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.601688    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601787    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601806    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:35.601912 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.601821    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:35.602487 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.601975    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:35.686562 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.686525    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.741085 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.740677    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal_457f0a554379acbff3f8f5b8e1c5a53f/setup/0.log"
Oct 30 20:37:35.741085 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.740737    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerStarted","Data":"4414e9acc09dbc9aefe8c3ef2c6cc77b9cadb27e4e2dc7d807871965d14b9a0c"}
Oct 30 20:37:35.741085 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.740877    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:35.751274 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.750363    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:35.751274 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.750415    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:35.751274 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:35.750432    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:35.751274 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.750751    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:35.786882 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.786836    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.887085 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.886996    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:35.987475 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:35.987439    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.088565 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.088531    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.193409 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.193313    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.293466 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.293429    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.393552 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.393516    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.494279 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.494203    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.594563 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.594520    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.660975 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.660944    2580 kuberuntime_manager.go:1707] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-qhl7c_openshift-multus(a0b8f9f1-3e33-40b5-bcc2-743909a0295f): CreateContainerConfigError: services have not yet been read at least once, cannot construct envvars" logger="UnhandledError"
Oct 30 20:37:36.661099 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.660987    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with CreateContainerConfigError: \"services have not yet been read at least once, cannot construct envvars\"" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" podUID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f"
Oct 30 20:37:36.694992 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.694957    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.747551 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.747483    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:36.752410 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.751590    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:36.752410 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.751626    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:36.752410 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.751640    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:36.757455 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.757176    2580 kuberuntime_manager.go:1707] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-qhl7c_openshift-multus(a0b8f9f1-3e33-40b5-bcc2-743909a0295f): CreateContainerConfigError: services have not yet been read at least once, cannot construct envvars" logger="UnhandledError"
Oct 30 20:37:36.757455 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.757222    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with CreateContainerConfigError: \"services have not yet been read at least once, cannot construct envvars\"" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" podUID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f"
Oct 30 20:37:36.795091 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.795052    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.857966 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.857941    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:36.860127 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.860078    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:36.860127 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.860119    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:36.860297 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:36.860135    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:36.895291 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.895259    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:36.995965 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:36.995924    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.096905 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.096869    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.197931 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.197887    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.233046 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.233013    2580 kuberuntime_manager.go:1707] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-bsg94_kube-system(fd3db454-59fb-46b2-90e9-3b7f4c11d5bb): CreateContainerConfigError: services have not yet been read at least once, cannot construct envvars" logger="UnhandledError"
Oct 30 20:37:37.233191 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.233058    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with CreateContainerConfigError: \"services have not yet been read at least once, cannot construct envvars\"" pod="kube-system/konnectivity-agent-bsg94" podUID="fd3db454-59fb-46b2-90e9-3b7f4c11d5bb"
Oct 30 20:37:37.260712 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.260671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:37.260871 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.260830    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.263482 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.263458    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.263626 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.263495    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.263626 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.263511    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.263739 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.263647    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:37.263739 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.263697    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs podName:d5a8e9f5-484c-4848-a012-21b8ec6ecb32 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:41.263677819 +0000 UTC m=+21.171986071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs") pod "network-metrics-daemon-p52s5" (UID: "d5a8e9f5-484c-4848-a012-21b8ec6ecb32") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:37.298990 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.298946    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.399548 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.399475    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.462945 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.462551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:37.462945 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.462732    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.465225 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.464298    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.465225 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.464357    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.465225 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.464375    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.465225 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.464536    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:37.465225 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.464591    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret podName:754d8def-099b-4fdb-859c-0f318bf5fd38 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:41.464571105 +0000 UTC m=+21.372879349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret") pod "global-pull-secret-syncer-tck7z" (UID: "754d8def-099b-4fdb-859c-0f318bf5fd38") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:37.485441 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.485409    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:37:37.499716 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.499673    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.512596 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.512311    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.514868 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.513948    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.514868 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.513982    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.514868 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.513999    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.564932 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.563606    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qfl7v"]
Oct 30 20:37:37.564932 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.564102    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.564932 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.564610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:37.564932 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.564844    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.567024 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.566290    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.567024 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.566321    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.567024 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.566336    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.567948 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.567374    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.567948 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.567413    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.567948 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.567428    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.567948 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.567560    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:37:37.567948 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.567574    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:37:37.567948 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.567586    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5jggs for pod openshift-network-diagnostics/network-check-target-lhg4p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:37.567948 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.567642    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs podName:0ca03c7b-5428-4123-980b-9bdbff49850f nodeName:}" failed. No retries permitted until 2026-10-30 20:37:41.567617822 +0000 UTC m=+21.475926064 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5jggs" (UniqueName: "kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs") pod "network-check-target-lhg4p" (UID: "0ca03c7b-5428-4123-980b-9bdbff49850f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:37.574917 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.574675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.574917 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.574713    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.576469 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.576105    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.576469 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.576134    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.576469 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.576148    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.582347 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.582325    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 20:37:37.582563 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.582545    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5c74x\""
Oct 30 20:37:37.582650 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.582599    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 20:37:37.599241 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.599212    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:37.599241 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.599244    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.599441 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.599300    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:37.599441 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.599332    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.599441 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.599370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:37.599441 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.599404    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.599801 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.599762    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.600636 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.600619    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.600722 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.600649    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.600722 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.600665    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.600897 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.600875    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:37.601489 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.601471    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.601551 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.601500    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.601551 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.601516    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.601685 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.601666    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:37.601770 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.601758    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.601817 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.601775    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.601817 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.601784    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.601901 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.601877    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:37.620035 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.620007    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 20:37:37.620158 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.620126    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 20:37:37.620271 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.620255    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 20:37:37.620351 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.620339    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 20:37:37.665137 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665137 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6mhd\" (UniqueName: \"kubernetes.io/projected/70f4b787-e9c9-4e85-894d-27d370f8766d-kube-api-access-h6mhd\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-sys\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-metrics-client-ca\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-textfile\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-root\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-tls\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665629 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-wtmp\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.665629 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.665374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.700814 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.700764    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.750262 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.750004    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.752636 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.751531    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.752636 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.751564    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.752636 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.751578    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.756401 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.755548    2580 kuberuntime_manager.go:1707] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-bsg94_kube-system(fd3db454-59fb-46b2-90e9-3b7f4c11d5bb): CreateContainerConfigError: services have not yet been read at least once, cannot construct envvars" logger="UnhandledError"
Oct 30 20:37:37.756401 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.755589    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with CreateContainerConfigError: \"services have not yet been read at least once, cannot construct envvars\"" pod="kube-system/konnectivity-agent-bsg94" podUID="fd3db454-59fb-46b2-90e9-3b7f4c11d5bb"
Oct 30 20:37:37.760846 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.760813    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-root\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-tls\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-wtmp\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6mhd\" (UniqueName: \"kubernetes.io/projected/70f4b787-e9c9-4e85-894d-27d370f8766d-kube-api-access-h6mhd\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-sys\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-metrics-client-ca\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.765969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-textfile\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.766302    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-textfile\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.766396    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-root\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.766475    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.766744    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-wtmp\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.766819    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.767361    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.767462    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.767512    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-sys\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.769406 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.768031    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-metrics-client-ca\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.771947    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.771980    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.771997    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.772620    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.772642    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.772656    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.773058    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.773077    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.773459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.773091    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.777450 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.777427    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.781838 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.781811    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-tls\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.786107 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.786079    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6mhd\" (UniqueName: \"kubernetes.io/projected/70f4b787-e9c9-4e85-894d-27d370f8766d-kube-api-access-h6mhd\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.801955 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.801902    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.867110 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.867076    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:37:37.869304 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.869265    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:37:37.869304 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.869300    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:37:37.869523 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.869317    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:37:37.900414 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:37.900352    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:37:37.902883 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.902834    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:37.930333 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:37.930296    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f4b787_e9c9_4e85_894d_27d370f8766d.slice/crio-76d66980b12fab770bd9ac58acb250d5183cad1ab6ddd94ca1596472fdadec7b WatchSource:0}: Error finding container 76d66980b12fab770bd9ac58acb250d5183cad1ab6ddd94ca1596472fdadec7b: Status 404 returned error can't find the container with id 76d66980b12fab770bd9ac58acb250d5183cad1ab6ddd94ca1596472fdadec7b
Oct 30 20:37:37.994958 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.994927    2580 kuberuntime_manager.go:1707] "Unhandled Error" err="container node-ca start failed in pod node-ca-29897_openshift-image-registry(4f7516cb-f8ee-4e50-88ae-f2cd6cd40087): CreateContainerConfigError: services have not yet been read at least once, cannot construct envvars" logger="UnhandledError"
Oct 30 20:37:37.995078 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:37.994976    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"node-ca\" with CreateContainerConfigError: \"services have not yet been read at least once, cannot construct envvars\"" pod="openshift-image-registry/node-ca-29897" podUID="4f7516cb-f8ee-4e50-88ae-f2cd6cd40087"
Oct 30 20:37:38.003453 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:38.003419    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:38.104319 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:38.104287    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:38.204628 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:38.204549    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:38.208926 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.208902    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:37:38.305044 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:38.305002    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:38.405644 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:38.405600    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:37:38.450660 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.450586    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:37:38.460736 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.460643    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:38.467074 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.467051    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 20:37:38.484563 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.484539    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:37:38.484678 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.484669    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:37:38.489450 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.485202    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"]
Oct 30 20:37:38.489450 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.488520    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 20:37:38.496833 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.496801    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:37:38.498724 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.498707    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"]
Oct 30 20:37:38.510470 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.510447    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g986l"
Oct 30 20:37:38.518583 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.518561    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g986l"
Oct 30 20:37:38.757728 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.757682    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qfl7v" event={"ID":"70f4b787-e9c9-4e85-894d-27d370f8766d","Type":"ContainerStarted","Data":"76d66980b12fab770bd9ac58acb250d5183cad1ab6ddd94ca1596472fdadec7b"}
Oct 30 20:37:38.811195 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:38.811142    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal" podStartSLOduration=0.811127372 podStartE2EDuration="811.127372ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:38.794189027 +0000 UTC m=+18.702497288" watchObservedRunningTime="2026-10-30 20:37:38.811127372 +0000 UTC m=+18.719435633"
Oct 30 20:37:39.525595 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.525550    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:32:38 +0000 UTC" deadline="2028-04-05 22:48:16.686046173 +0000 UTC"
Oct 30 20:37:39.525595 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.525595    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12554h10m37.160458585s"
Oct 30 20:37:39.598928 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.598851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:39.599095 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:39.598979    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:39.599693 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.599401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:39.599693 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:39.599504    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:39.599693 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.599563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:39.599693 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:39.599648    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:39.762983 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.762024    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal_457f0a554379acbff3f8f5b8e1c5a53f/setup/0.log"
Oct 30 20:37:39.762983 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.762066    2580 generic.go:356] "Generic (PLEG): container finished" podID="457f0a554379acbff3f8f5b8e1c5a53f" containerID="4414e9acc09dbc9aefe8c3ef2c6cc77b9cadb27e4e2dc7d807871965d14b9a0c" exitCode=0
Oct 30 20:37:39.762983 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.762148    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerDied","Data":"4414e9acc09dbc9aefe8c3ef2c6cc77b9cadb27e4e2dc7d807871965d14b9a0c"}
Oct 30 20:37:39.762983 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.762206    2580 scope.go:122] "RemoveContainer" containerID="10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246"
Oct 30 20:37:39.762983 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.762641    2580 scope.go:122] "RemoveContainer" containerID="10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246"
Oct 30 20:37:39.766542 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.766421    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt" event={"ID":"a8ff15b4-7b0f-4e99-883c-05938faa37df","Type":"ContainerStarted","Data":"d33d647a669c29ddd1a0643b754de0eeb325a0e0640d347e601e1e7fc0cabb62"}
Oct 30 20:37:39.770165 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:39.769764    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-29897" event={"ID":"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087","Type":"ContainerStarted","Data":"607be5307e84d294e546902b6334f7f52962fa6dd8962fa1684b2dfbec8b0ed1"}
Oct 30 20:37:39.782832 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:39.782790    2580 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal_openshift-machine-config-operator_457f0a554379acbff3f8f5b8e1c5a53f_0 in pod sandbox 8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c from index: no such id: '10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246'" containerID="10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246"
Oct 30 20:37:39.782966 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:39.782855    2580 kuberuntime_container.go:987] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal_openshift-machine-config-operator_457f0a554379acbff3f8f5b8e1c5a53f_0 in pod sandbox 8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c from index: no such id: '10a4c56b4fe7afa665d075d0888776c0960efa3d6d7e17b247349f7382363246'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal_openshift-machine-config-operator(457f0a554379acbff3f8f5b8e1c5a53f)\"" logger="UnhandledError"
Oct 30 20:37:40.532190 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.531724    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:32:38 +0000 UTC" deadline="2028-05-05 19:19:30.762873122 +0000 UTC"
Oct 30 20:37:40.532190 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.531992    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13270h41m50.230889318s"
Oct 30 20:37:40.780432 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.780372    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerStarted","Data":"a3212373aeca09679f1af89b0027024ff394fe48f4b3bb46ab1db5922f3e51be"}
Oct 30 20:37:40.783494 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.783424    2580 generic.go:356] "Generic (PLEG): container finished" podID="70f4b787-e9c9-4e85-894d-27d370f8766d" containerID="60eae345a6153e47be7a77dcb24c9c282a2d4af9744389a065f12a97941e09da" exitCode=0
Oct 30 20:37:40.783494 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.783475    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qfl7v" event={"ID":"70f4b787-e9c9-4e85-894d-27d370f8766d","Type":"ContainerDied","Data":"60eae345a6153e47be7a77dcb24c9c282a2d4af9744389a065f12a97941e09da"}
Oct 30 20:37:40.797484 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.797429    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-29897" podStartSLOduration=3.512026761 podStartE2EDuration="7.797413154s" totalImagesPullingTime="4.285386393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:39.809401906 +0000 UTC m=+19.717710160" watchObservedRunningTime="2026-10-30 20:37:40.797413154 +0000 UTC m=+20.705721415"
Oct 30 20:37:40.821978 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.821908    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" podStartSLOduration=2.821888625 podStartE2EDuration="2.821888625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:40.79829225 +0000 UTC m=+20.706600515" watchObservedRunningTime="2026-10-30 20:37:40.821888625 +0000 UTC m=+20.730196887"
Oct 30 20:37:40.886761 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:40.885413    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 20:37:41.298462 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.298422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:41.298646 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.298603    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:41.298708 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.298666    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs podName:d5a8e9f5-484c-4848-a012-21b8ec6ecb32 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:49.298648572 +0000 UTC m=+29.206956816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs") pod "network-metrics-daemon-p52s5" (UID: "d5a8e9f5-484c-4848-a012-21b8ec6ecb32") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:41.500024 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.499988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:41.500754 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.500197    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:41.500754 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.500265    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret podName:754d8def-099b-4fdb-859c-0f318bf5fd38 nodeName:}" failed. No retries permitted until 2026-10-30 20:37:49.500247087 +0000 UTC m=+29.408555347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret") pod "global-pull-secret-syncer-tck7z" (UID: "754d8def-099b-4fdb-859c-0f318bf5fd38") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:41.533714 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.532822    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T20:37:40.885435101Z","UUID":"7f9f81a6-8daa-41c4-a1ac-47556170b46d","Handler":null,"Name":"","Endpoint":""}
Oct 30 20:37:41.535125 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.535103    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 20:37:41.535249 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.535136    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 20:37:41.599146 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.599112    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:41.599266 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.599229    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:41.599678 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.599658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:41.599788 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.599770    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:41.599841 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.599826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:41.599908 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.599893    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:41.601303 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.601276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:41.601506 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.601487    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:37:41.601574 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.601515    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:37:41.601574 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.601531    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5jggs for pod openshift-network-diagnostics/network-check-target-lhg4p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:41.601676 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:41.601585    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs podName:0ca03c7b-5428-4123-980b-9bdbff49850f nodeName:}" failed. No retries permitted until 2026-10-30 20:37:49.60156554 +0000 UTC m=+29.509873799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5jggs" (UniqueName: "kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs") pod "network-check-target-lhg4p" (UID: "0ca03c7b-5428-4123-980b-9bdbff49850f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:41.788496 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.788459    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qfl7v" event={"ID":"70f4b787-e9c9-4e85-894d-27d370f8766d","Type":"ContainerStarted","Data":"42b705f06fa7c5ce477a4c2b0298418ecc697c80a3d2c4d02168f34122d22623"}
Oct 30 20:37:41.788664 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.788520    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qfl7v" event={"ID":"70f4b787-e9c9-4e85-894d-27d370f8766d","Type":"ContainerStarted","Data":"35cb70e45252afac0f5cab1734a709165d459ad2b0b62221abe4505fc5fbe71f"}
Oct 30 20:37:41.791347 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.791317    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f7tx2" event={"ID":"f0d61b27-34af-40d5-8ab0-b06c28aa86e4","Type":"ContainerStarted","Data":"ae1c67b1a2a2c3cfb6395125b409bf3fdefaa1d49dc21af18bd80c226e7c8943"}
Oct 30 20:37:41.794722 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.794634    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt" event={"ID":"a8ff15b4-7b0f-4e99-883c-05938faa37df","Type":"ContainerStarted","Data":"961c46a213ccb732bfd6cd351b81187fa68b40e8948f801ea4a1d5432d64ac8b"}
Oct 30 20:37:41.797612 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.797120    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g9hwf" event={"ID":"9c7b0506-8e4a-43bd-b7a6-e58e9879d893","Type":"ContainerStarted","Data":"d1fdd56556d5f43d351c61cd033a0533270ba96d30d3e9f48911e9ac445826f9"}
Oct 30 20:37:41.809401 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.809348    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qfl7v" podStartSLOduration=2.8758350029999997 podStartE2EDuration="4.809336887s" totalImagesPullingTime="1.831720291s" totalInitContainerRuntime="101.781593ms" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:41.809009739 +0000 UTC m=+21.717318014" watchObservedRunningTime="2026-10-30 20:37:41.809336887 +0000 UTC m=+21.717645149"
Oct 30 20:37:41.828573 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.828524    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-f7tx2" podStartSLOduration=1.544790066 podStartE2EDuration="8.828512358s" totalImagesPullingTime="7.283722292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:41.828176347 +0000 UTC m=+21.736484610" watchObservedRunningTime="2026-10-30 20:37:41.828512358 +0000 UTC m=+21.736820609"
Oct 30 20:37:41.843570 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:41.843518    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-g9hwf" podStartSLOduration=1.439067303 podStartE2EDuration="8.843501643s" totalImagesPullingTime="7.40443434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:41.842901438 +0000 UTC m=+21.751209700" watchObservedRunningTime="2026-10-30 20:37:41.843501643 +0000 UTC m=+21.751809905"
Oct 30 20:37:42.801471 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:42.801430    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt" event={"ID":"a8ff15b4-7b0f-4e99-883c-05938faa37df","Type":"ContainerStarted","Data":"5192521a6de6b07d4d1cfa4646d943e0f026c588aadefa37659671dbe090dc45"}
Oct 30 20:37:43.598624 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.598359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:43.598776 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.598359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:43.598776 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.598424    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:43.598776 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:43.598691    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:43.598900 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:43.598798    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:43.598900 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:43.598885    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:43.805515 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.805471    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2rrmv" event={"ID":"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0","Type":"ContainerStarted","Data":"c2a70213fe6f83551eae9b8171bc64c346c4c302c56c19cf49f638eccc9128fa"}
Oct 30 20:37:43.807241 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.807212    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52cfv" event={"ID":"00295160-ccc3-481f-ac98-7b9e15aa7c25","Type":"ContainerStarted","Data":"918ce4f94a67a1ac684cba191f407d44753dbff9f7640a141e39fd2fcec3a39f"}
Oct 30 20:37:43.822470 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.822421    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt" podStartSLOduration=2.9896455790000003 podStartE2EDuration="10.822406731s" totalImagesPullingTime="7.832761152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:42.837564062 +0000 UTC m=+22.745872324" watchObservedRunningTime="2026-10-30 20:37:43.822406731 +0000 UTC m=+23.730714995"
Oct 30 20:37:43.822765 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.822739    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2rrmv" podStartSLOduration=1.46908605 podStartE2EDuration="10.822729576s" totalImagesPullingTime="9.353643526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:43.822014777 +0000 UTC m=+23.730323040" watchObservedRunningTime="2026-10-30 20:37:43.822729576 +0000 UTC m=+23.731037841"
Oct 30 20:37:43.836870 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:43.836817    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-52cfv" podStartSLOduration=3.150773541 podStartE2EDuration="10.836802733s" totalImagesPullingTime="7.686029192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:43.836449744 +0000 UTC m=+23.744758006" watchObservedRunningTime="2026-10-30 20:37:43.836802733 +0000 UTC m=+23.745110995"
Oct 30 20:37:45.599087 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:45.599048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:45.599607 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:45.599071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:45.599607 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:45.599172    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:45.599607 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:45.599201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:45.599607 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:45.599311    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:45.599607 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:45.599379    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:47.598863 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.598824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:47.598863 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.598851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:47.599342 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.598855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:47.599342 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:47.598931    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:47.599342 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:47.599010    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:47.599342 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:47.599088    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:47.819311 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.819275    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"ce7d03ffd60ae6416722f0fb13be780d4887564b87853a32a190b7380dcd2962"}
Oct 30 20:37:47.819491 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.819356    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"e50537e6b69f5617c2b0441b98c95257dd478c07a83051525ebbde8b0a970f78"}
Oct 30 20:37:47.819491 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.819414    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"ea84e3b923cf5e836ccef2c9cb2002ebcee96de75ae31a506b2b8ef9959363a3"}
Oct 30 20:37:47.819491 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.819456    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"fa11e0947a187d0f79563666062cf4e135e01e368b76abc955ecc4368511bf26"}
Oct 30 20:37:47.819686 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.819495    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"1615eb07126a0f6381054fddbb884ced7cdc76b78effc81b0351861e1565c42e"}
Oct 30 20:37:47.819686 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:47.819542    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"4f35f27b68af0ea21859856bd081dfd6399517f127c7f29a58dfa02db66feeb0"}
Oct 30 20:37:49.357732 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.357704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:49.358156 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.357828    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:49.358156 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.357894    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs podName:d5a8e9f5-484c-4848-a012-21b8ec6ecb32 nodeName:}" failed. No retries permitted until 2026-10-30 20:38:05.357875011 +0000 UTC m=+45.266183263 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs") pod "network-metrics-daemon-p52s5" (UID: "d5a8e9f5-484c-4848-a012-21b8ec6ecb32") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 20:37:49.559075 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.559038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:49.559264 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.559199    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:49.559333 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.559272    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret podName:754d8def-099b-4fdb-859c-0f318bf5fd38 nodeName:}" failed. No retries permitted until 2026-10-30 20:38:05.559252988 +0000 UTC m=+45.467561227 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret") pod "global-pull-secret-syncer-tck7z" (UID: "754d8def-099b-4fdb-859c-0f318bf5fd38") : object "kube-system"/"original-pull-secret" not registered
Oct 30 20:37:49.598569 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.598486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:49.598569 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.598550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:49.598737 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.598648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:49.598737 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.598657    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:49.598737 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.598728    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:49.598865 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.598796    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:49.660079 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.660051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:49.660221 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.660204    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 20:37:49.660276 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.660227    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 20:37:49.660276 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.660239    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5jggs for pod openshift-network-diagnostics/network-check-target-lhg4p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:49.660338 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:49.660313    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs podName:0ca03c7b-5428-4123-980b-9bdbff49850f nodeName:}" failed. No retries permitted until 2026-10-30 20:38:05.660298999 +0000 UTC m=+45.568607259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5jggs" (UniqueName: "kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs") pod "network-check-target-lhg4p" (UID: "0ca03c7b-5428-4123-980b-9bdbff49850f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 20:37:49.825467 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.825434    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="967a1dedee09acbcdf8d02323de68e24a22c8676a1ce123744d8c70e0127265b" exitCode=0
Oct 30 20:37:49.825599 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.825506    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerDied","Data":"967a1dedee09acbcdf8d02323de68e24a22c8676a1ce123744d8c70e0127265b"}
Oct 30 20:37:49.829039 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:49.829019    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"f52f65a7afd5904595917b8a70a1193f420aaa2ab5858e9f39bfe37c8d5f07ae"}
Oct 30 20:37:50.833194 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:50.833158    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bsg94" event={"ID":"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb","Type":"ContainerStarted","Data":"abfe12666c3d4ee6326ae57efd971884b4f7e8ffb94fbdb45b8c24c906bab4d2"}
Oct 30 20:37:50.848377 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:50.848326    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bsg94" podStartSLOduration=14.326943517 podStartE2EDuration="17.848308254s" totalImagesPullingTime="3.521364737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:50.847935335 +0000 UTC m=+30.756243598" watchObservedRunningTime="2026-10-30 20:37:50.848308254 +0000 UTC m=+30.756616517"
Oct 30 20:37:50.891301 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:50.891226    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:50.973161 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:50.973121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:51.598595 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.598327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:51.598595 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.598572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:51.598857 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.598530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:51.598857 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:51.598688    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:51.598857 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:51.598772    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:51.598857 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:51.598849    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:51.837611 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.837569    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerStarted","Data":"a3739d963adf3b79f87d3a8fc59ffafa3f8fe84df4858227fa054ab0b8bfb7eb"}
Oct 30 20:37:51.841473 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.841448    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" event={"ID":"260a13a6-e92d-4243-991e-ba4639127adc","Type":"ContainerStarted","Data":"2a962ed3c61434d83c609f5f30d1f079b70c6c67953d8d26491871d3a93f246e"}
Oct 30 20:37:51.841811 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.841785    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:51.841940 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.841924    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:51.842009 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.841996    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:51.857299 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.857271    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:51.857512 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.857433    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:37:51.890467 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:51.890350    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-825zz" podStartSLOduration=5.691249626 podStartE2EDuration="18.890336854s" totalImagesPullingTime="13.199087228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:37:51.885442849 +0000 UTC m=+31.793751110" watchObservedRunningTime="2026-10-30 20:37:51.890336854 +0000 UTC m=+31.798645114"
Oct 30 20:37:52.046429 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:52.046406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:52.147592 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:52.147562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:52.846627 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:52.846592    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="a3739d963adf3b79f87d3a8fc59ffafa3f8fe84df4858227fa054ab0b8bfb7eb" exitCode=0
Oct 30 20:37:52.847202 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:52.846674    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerDied","Data":"a3739d963adf3b79f87d3a8fc59ffafa3f8fe84df4858227fa054ab0b8bfb7eb"}
Oct 30 20:37:52.962362 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:52.962335    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:53.065026 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.064985    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:53.143199 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.143131    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tck7z"]
Oct 30 20:37:53.143337 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.143252    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:53.143379 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:53.143336    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:53.146954 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.146928    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lhg4p"]
Oct 30 20:37:53.147100 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.146979    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p52s5"]
Oct 30 20:37:53.147100 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.147052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:53.147220 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:53.147127    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:53.147220 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.147184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:53.147318 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:53.147241    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:53.158163 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.158138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:53.228556 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.228511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:53.699935 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.699727    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:53.700415 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.700371    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:53.713274 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.713250    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:53.851624 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.851541    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="2202412eae9a7f879c07f99c821823fca95e81b2bfe7a698b02cb0d28e43bc3f" exitCode=0
Oct 30 20:37:53.851624 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.851567    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerDied","Data":"2202412eae9a7f879c07f99c821823fca95e81b2bfe7a698b02cb0d28e43bc3f"}
Oct 30 20:37:53.852184 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.852147    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:53.853277 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.852885    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:37:53.863162 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.863131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:53.959914 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:53.959884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:54.420297 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:54.420265    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:54.520067 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:54.520036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:54.599309 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:54.599278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:54.599472 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:54.599279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:54.599472 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:54.599379    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:54.599472 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:54.599399    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:54.599638 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:54.599494    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:54.599638 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:54.599557    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:54.605168 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:54.605138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:54.918159 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:54.918130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:55.022872 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:55.022838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:55.858693 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:55.858659    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="2f951ba6b578551ff6e991b6c8baad203619db7f82ac2e1b92f01983a917f64c" exitCode=0
Oct 30 20:37:55.858852 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:55.858728    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerDied","Data":"2f951ba6b578551ff6e991b6c8baad203619db7f82ac2e1b92f01983a917f64c"}
Oct 30 20:37:56.307210 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:56.307136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:37:56.598724 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:56.598637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:56.598891 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:56.598644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:56.598891 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:56.598769    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:56.598891 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:56.598791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:56.599047 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:56.598905    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:56.599047 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:56.599023    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:57.866818 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:57.866787    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerStarted","Data":"8d339f08ef793c0d03283994a7f62ff7f2d045665555edc3f8123a360ee9edaa"}
Oct 30 20:37:58.598668 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.598632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:37:58.598855 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.598636    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:37:58.598855 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:58.598779    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p52s5" podUID="d5a8e9f5-484c-4848-a012-21b8ec6ecb32"
Oct 30 20:37:58.598855 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.598636    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:37:58.598855 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:58.598842    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tck7z" podUID="754d8def-099b-4fdb-859c-0f318bf5fd38"
Oct 30 20:37:58.599005 ip-10-0-135-178 kubenswrapper[2580]: E1030 20:37:58.598897    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lhg4p" podUID="0ca03c7b-5428-4123-980b-9bdbff49850f"
Oct 30 20:37:58.842935 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.842905    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeReady"
Oct 30 20:37:58.843111 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.843062    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 20:37:58.872223 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.872193    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="8d339f08ef793c0d03283994a7f62ff7f2d045665555edc3f8123a360ee9edaa" exitCode=0
Oct 30 20:37:58.872675 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.872254    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerDied","Data":"8d339f08ef793c0d03283994a7f62ff7f2d045665555edc3f8123a360ee9edaa"}
Oct 30 20:37:58.936698 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.936668    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mrq2q"]
Oct 30 20:37:58.954615 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.954593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:58.966154 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.966126    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 20:37:58.966275 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.966225    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 20:37:58.966764 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.966575    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 20:37:58.966764 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.966734    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rsx8b\""
Oct 30 20:37:58.967108 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:58.967083    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mrq2q"]
Oct 30 20:37:59.033636 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.033606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-cert\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:59.033772 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.033667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qklrf\" (UniqueName: \"kubernetes.io/projected/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-kube-api-access-qklrf\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:59.061544 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.061512    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rjlsv"]
Oct 30 20:37:59.069403 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.069369    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ck9ws"]
Oct 30 20:37:59.069507 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.069488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.078428 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.078402    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 20:37:59.078560 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.078450    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qtwx7\""
Oct 30 20:37:59.078560 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.078464    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 20:37:59.078671 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.078604    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 20:37:59.078671 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.078646    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 20:37:59.079410 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.079376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.079623 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.079606    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rjlsv"]
Oct 30 20:37:59.087004 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.086979    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ck9ws"]
Oct 30 20:37:59.134655 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qklrf\" (UniqueName: \"kubernetes.io/projected/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-kube-api-access-qklrf\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:59.134655 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wb89\" (UniqueName: \"kubernetes.io/projected/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-api-access-5wb89\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.134849 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14627328-5fa1-4c44-93a1-94b03f420bc7-config-volume\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.134849 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wds6s\" (UniqueName: \"kubernetes.io/projected/14627328-5fa1-4c44-93a1-94b03f420bc7-kube-api-access-wds6s\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.134849 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5108533b-2d1b-488f-a03c-fd5beb4f34f5-crio-socket\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.134849 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5108533b-2d1b-488f-a03c-fd5beb4f34f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.134849 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-cert\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:59.134849 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14627328-5fa1-4c44-93a1-94b03f420bc7-tmp-dir\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.135172 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.135172 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.134982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14627328-5fa1-4c44-93a1-94b03f420bc7-metrics-tls\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.135172 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.135006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5108533b-2d1b-488f-a03c-fd5beb4f34f5-data-volume\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.138910 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.138885    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-cert\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:59.144282 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.144260    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qklrf\" (UniqueName: \"kubernetes.io/projected/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-kube-api-access-qklrf\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:59.178447 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.178419    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 20:37:59.178447 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.178419    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-69hq7\""
Oct 30 20:37:59.178874 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.178850    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 20:37:59.235885 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.235849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wb89\" (UniqueName: \"kubernetes.io/projected/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-api-access-5wb89\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.236035 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.235898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14627328-5fa1-4c44-93a1-94b03f420bc7-config-volume\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.236035 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.235922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wds6s\" (UniqueName: \"kubernetes.io/projected/14627328-5fa1-4c44-93a1-94b03f420bc7-kube-api-access-wds6s\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.236035 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.235977    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5108533b-2d1b-488f-a03c-fd5beb4f34f5-crio-socket\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.236035 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5108533b-2d1b-488f-a03c-fd5beb4f34f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.236255 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14627328-5fa1-4c44-93a1-94b03f420bc7-tmp-dir\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.236255 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.236255 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14627328-5fa1-4c44-93a1-94b03f420bc7-metrics-tls\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.236255 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5108533b-2d1b-488f-a03c-fd5beb4f34f5-data-volume\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.236482 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236273    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5108533b-2d1b-488f-a03c-fd5beb4f34f5-crio-socket\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.236529 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236498    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14627328-5fa1-4c44-93a1-94b03f420bc7-tmp-dir\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.236585 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236566    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5108533b-2d1b-488f-a03c-fd5beb4f34f5-data-volume\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.236715 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236694    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14627328-5fa1-4c44-93a1-94b03f420bc7-config-volume\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.236819 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.236800    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.238370 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.238349    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14627328-5fa1-4c44-93a1-94b03f420bc7-metrics-tls\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.238475 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.238457    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5108533b-2d1b-488f-a03c-fd5beb4f34f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.247692 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.247665    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wb89\" (UniqueName: \"kubernetes.io/projected/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-api-access-5wb89\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.247782 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.247714    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wds6s\" (UniqueName: \"kubernetes.io/projected/14627328-5fa1-4c44-93a1-94b03f420bc7-kube-api-access-wds6s\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.269691 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.269660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:37:59.405045 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.404944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:37:59.418729 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.418701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:37:59.590400 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.590344    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mrq2q"]
Oct 30 20:37:59.594362 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:59.594329    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf314cf90_719e_4d6d_8745_0ff46bbbf9f8.slice/crio-0e7445d7143a33f386eea8bf81a5da02f863bcf5e074627b7317e219ea5a90e6 WatchSource:0}: Error finding container 0e7445d7143a33f386eea8bf81a5da02f863bcf5e074627b7317e219ea5a90e6: Status 404 returned error can't find the container with id 0e7445d7143a33f386eea8bf81a5da02f863bcf5e074627b7317e219ea5a90e6
Oct 30 20:37:59.699020 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.698943    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rjlsv"]
Oct 30 20:37:59.702475 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:59.702448    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5108533b_2d1b_488f_a03c_fd5beb4f34f5.slice/crio-904b70662e4a98f7dd9ceeda0c56a881389cbf09cf1fb81b31aa0bd60dece6d1 WatchSource:0}: Error finding container 904b70662e4a98f7dd9ceeda0c56a881389cbf09cf1fb81b31aa0bd60dece6d1: Status 404 returned error can't find the container with id 904b70662e4a98f7dd9ceeda0c56a881389cbf09cf1fb81b31aa0bd60dece6d1
Oct 30 20:37:59.718229 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.718209    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ck9ws"]
Oct 30 20:37:59.723272 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:37:59.723248    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14627328_5fa1_4c44_93a1_94b03f420bc7.slice/crio-be06a115b0611fba33b94f8e57f95e70203f8ccf3344f0a83b00279b89738ae0 WatchSource:0}: Error finding container be06a115b0611fba33b94f8e57f95e70203f8ccf3344f0a83b00279b89738ae0: Status 404 returned error can't find the container with id be06a115b0611fba33b94f8e57f95e70203f8ccf3344f0a83b00279b89738ae0
Oct 30 20:37:59.877626 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.877592    2580 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="32c600acbade84541cd27bd718393442e516dff63b2c510dc184cc605e331b40" exitCode=0
Oct 30 20:37:59.878166 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.877676    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerDied","Data":"32c600acbade84541cd27bd718393442e516dff63b2c510dc184cc605e331b40"}
Oct 30 20:37:59.879237 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.879213    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjlsv" event={"ID":"5108533b-2d1b-488f-a03c-fd5beb4f34f5","Type":"ContainerStarted","Data":"671b600a4aebce0d2ca97ca206a88389b97390dddc4ba27f2917c8df5e8909b2"}
Oct 30 20:37:59.879306 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.879266    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjlsv" event={"ID":"5108533b-2d1b-488f-a03c-fd5beb4f34f5","Type":"ContainerStarted","Data":"904b70662e4a98f7dd9ceeda0c56a881389cbf09cf1fb81b31aa0bd60dece6d1"}
Oct 30 20:37:59.880421 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.880371    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mrq2q" event={"ID":"f314cf90-719e-4d6d-8745-0ff46bbbf9f8","Type":"ContainerStarted","Data":"0e7445d7143a33f386eea8bf81a5da02f863bcf5e074627b7317e219ea5a90e6"}
Oct 30 20:37:59.881581 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:37:59.881559    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ck9ws" event={"ID":"14627328-5fa1-4c44-93a1-94b03f420bc7","Type":"ContainerStarted","Data":"be06a115b0611fba33b94f8e57f95e70203f8ccf3344f0a83b00279b89738ae0"}
Oct 30 20:38:00.601875 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.601693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:00.602054 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.601694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:00.602054 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.601749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:00.606238 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.606214    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 20:38:00.608747 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.608721    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 20:38:00.608869 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.608746    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 20:38:00.608869 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.608833    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 20:38:00.608869 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.608836    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zljsp\""
Oct 30 20:38:00.609012 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.608892    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-jrl7k\""
Oct 30 20:38:00.609012 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.608912    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-4pvnz\""
Oct 30 20:38:00.889142 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.889054    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" event={"ID":"a0b8f9f1-3e33-40b5-bcc2-743909a0295f","Type":"ContainerStarted","Data":"2804475fb43fc3d89c7c920f64d57f1d27d3e5e0938b17623d90745a77fc5a0e"}
Oct 30 20:38:00.912275 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:00.912214    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qhl7c" podStartSLOduration=19.888284274 podStartE2EDuration="27.912194428s" totalImagesPullingTime="8.023910154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:38:00.910850776 +0000 UTC m=+40.819159041" watchObservedRunningTime="2026-10-30 20:38:00.912194428 +0000 UTC m=+40.820502692"
Oct 30 20:38:01.037794 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:01.037578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:01.138838 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:01.138719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:01.237677 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:01.237592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:01.548135 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:01.548104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:01.895827 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:01.895738    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjlsv" event={"ID":"5108533b-2d1b-488f-a03c-fd5beb4f34f5","Type":"ContainerStarted","Data":"b4fb5d3140e126d10bd1e99c015140760da32c047276d8417e1bfe3b98a73210"}
Oct 30 20:38:02.662581 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:02.662535    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:02.867999 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:02.867970    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:02.899844 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:02.899818    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ck9ws" event={"ID":"14627328-5fa1-4c44-93a1-94b03f420bc7","Type":"ContainerStarted","Data":"b1bc251a1556560def3a71c268892e622a2a8fe7cd9deb49b2e0fb34d239dd10"}
Oct 30 20:38:02.900242 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:02.899863    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ck9ws" event={"ID":"14627328-5fa1-4c44-93a1-94b03f420bc7","Type":"ContainerStarted","Data":"a1f5485288710a16b797daf3299a0a9eccef7f3c8df598354c164eb4ebd02930"}
Oct 30 20:38:02.900242 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:02.899964    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:02.925253 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:02.925169    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-ck9ws" podStartSLOduration=2.58736848 podStartE2EDuration="4.925155821s" totalImagesPullingTime="2.337787341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:38:02.923755348 +0000 UTC m=+42.832063608" watchObservedRunningTime="2026-10-30 20:38:02.925155821 +0000 UTC m=+42.833464081"
Oct 30 20:38:03.906459 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:03.906426    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rjlsv" event={"ID":"5108533b-2d1b-488f-a03c-fd5beb4f34f5","Type":"ContainerStarted","Data":"51cb567095c733a72be9620d44860aa64baacd8c45d0ed9db0bf6de3fd6fd46f"}
Oct 30 20:38:03.908179 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:03.908154    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mrq2q" event={"ID":"f314cf90-719e-4d6d-8745-0ff46bbbf9f8","Type":"ContainerStarted","Data":"7428b8b272dcf732016b7a691646eca334900e31b430229dede06a0a14b7280f"}
Oct 30 20:38:03.935134 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:03.935089    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rjlsv" podStartSLOduration=2.438949575 podStartE2EDuration="5.935076708s" totalImagesPullingTime="3.496127133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:37:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:38:03.933286441 +0000 UTC m=+43.841594702" watchObservedRunningTime="2026-10-30 20:38:03.935076708 +0000 UTC m=+43.843384966"
Oct 30 20:38:03.961097 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:03.961044    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mrq2q" podStartSLOduration=2.192579349 podStartE2EDuration="5.961031023s" totalImagesPullingTime="3.768451674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:38:03.960791087 +0000 UTC m=+43.869099345" watchObservedRunningTime="2026-10-30 20:38:03.961031023 +0000 UTC m=+43.869339283"
Oct 30 20:38:03.973637 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:03.973610    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:04.152209 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:04.152174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:05.219445 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.219413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:05.332209 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.332178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:05.383571 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.383536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:05.387703 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.387678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:05.468158 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.468115    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:05.585160 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.585132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:05.587433 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.587410    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:05.659753 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.659717    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:05.686494 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.686467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:05.688993 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.688972    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:05.735846 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.735813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:05.761264 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.761236    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p52s5"]
Oct 30 20:38:05.764080 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:38:05.764056    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a8e9f5_484c_4848_a012_21b8ec6ecb32.slice/crio-13a39580dc8eda0b6b525d28c4572c5ca116594e91a392999d4e1cd70c972c50 WatchSource:0}: Error finding container 13a39580dc8eda0b6b525d28c4572c5ca116594e91a392999d4e1cd70c972c50: Status 404 returned error can't find the container with id 13a39580dc8eda0b6b525d28c4572c5ca116594e91a392999d4e1cd70c972c50
Oct 30 20:38:05.779239 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.779214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:05.914501 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:05.914465    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p52s5" event={"ID":"d5a8e9f5-484c-4848-a012-21b8ec6ecb32","Type":"ContainerStarted","Data":"13a39580dc8eda0b6b525d28c4572c5ca116594e91a392999d4e1cd70c972c50"}
Oct 30 20:38:06.034813 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.034736    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lhg4p"]
Oct 30 20:38:06.037827 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:38:06.037802    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ca03c7b_5428_4123_980b_9bdbff49850f.slice/crio-54ee5af7ec3419b96c81de00beb9b61a97c7e9388cfabbe603724387e4a0ac97 WatchSource:0}: Error finding container 54ee5af7ec3419b96c81de00beb9b61a97c7e9388cfabbe603724387e4a0ac97: Status 404 returned error can't find the container with id 54ee5af7ec3419b96c81de00beb9b61a97c7e9388cfabbe603724387e4a0ac97
Oct 30 20:38:06.079488 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.079466    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tck7z"]
Oct 30 20:38:06.082339 ip-10-0-135-178 kubenswrapper[2580]: W1030 20:38:06.082316    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod754d8def_099b_4fdb_859c_0f318bf5fd38.slice/crio-10cf589376990dc2b8cf887f858274dc52495014b70934ce59850591c23d95f8 WatchSource:0}: Error finding container 10cf589376990dc2b8cf887f858274dc52495014b70934ce59850591c23d95f8: Status 404 returned error can't find the container with id 10cf589376990dc2b8cf887f858274dc52495014b70934ce59850591c23d95f8
Oct 30 20:38:06.304654 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.304546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:06.404629 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.404593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:06.566237 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.566150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:06.665605 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.665567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:06.920773 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.920689    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tck7z" event={"ID":"754d8def-099b-4fdb-859c-0f318bf5fd38","Type":"ContainerStarted","Data":"10cf589376990dc2b8cf887f858274dc52495014b70934ce59850591c23d95f8"}
Oct 30 20:38:06.922451 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:06.922419    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lhg4p" event={"ID":"0ca03c7b-5428-4123-980b-9bdbff49850f","Type":"ContainerStarted","Data":"54ee5af7ec3419b96c81de00beb9b61a97c7e9388cfabbe603724387e4a0ac97"}
Oct 30 20:38:07.929929 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:07.929834    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p52s5" event={"ID":"d5a8e9f5-484c-4848-a012-21b8ec6ecb32","Type":"ContainerStarted","Data":"225a7c11325061bad33121a21af8344d898d4f8dd5852c3f83675f03fe031d98"}
Oct 30 20:38:07.929929 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:07.929906    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p52s5" event={"ID":"d5a8e9f5-484c-4848-a012-21b8ec6ecb32","Type":"ContainerStarted","Data":"525a892da3821e261e59855117c49b69cb4ffdf2f822035f7927d6be6ee9d758"}
Oct 30 20:38:07.957045 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:07.956993    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p52s5" podStartSLOduration=33.462406079 podStartE2EDuration="34.956977828s" totalImagesPullingTime="1.494571749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:38:07.955311262 +0000 UTC m=+47.863619651" watchObservedRunningTime="2026-10-30 20:38:07.956977828 +0000 UTC m=+47.865286089"
Oct 30 20:38:08.140802 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:08.140767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:08.241511 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:08.241412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:09.942056 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:09.942017    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lhg4p" event={"ID":"0ca03c7b-5428-4123-980b-9bdbff49850f","Type":"ContainerStarted","Data":"83e990aeefaaccbeab6695118cacfc6a339e8f3223e844ce7521dfd38a00c02f"}
Oct 30 20:38:09.942531 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:09.942193    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:09.960863 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:09.960801    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lhg4p" podStartSLOduration=33.182689503 podStartE2EDuration="36.96078095s" totalImagesPullingTime="3.778091447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:38:09.95924361 +0000 UTC m=+49.867551870" watchObservedRunningTime="2026-10-30 20:38:09.96078095 +0000 UTC m=+49.869089272"
Oct 30 20:38:11.826113 ip-10-0-135-178 kubenswrapper[2580]: I1030 20:38:11.826016    2580 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:38:11.826191 ip-10-0-135-178 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 20:38:11.838903 ip-10-0-135-178 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 20:38:11.839102 ip-10-0-135-178 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 20:38:11.839288 ip-10-0-135-178 systemd[1]: kubelet.service: Consumed 2.911s CPU time.
Oct 30 20:38:11.855937 ip-10-0-135-178 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 20:38:11.914116 ip-10-0-135-178 kubenswrapper[4852]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:38:11.914116 ip-10-0-135-178 kubenswrapper[4852]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 20:38:11.914116 ip-10-0-135-178 kubenswrapper[4852]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:38:11.914116 ip-10-0-135-178 kubenswrapper[4852]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 20:38:11.916365 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916290    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.916365 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916364    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.916365 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916367    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916371    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916374    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916377    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916380    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916416    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916420    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916423    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916426    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916428    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916431    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916434    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916437    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916441    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916449    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916452    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916455    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916458    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916461    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916464    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.916504 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916466    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916469    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916471    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916474    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916477    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916480    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916483    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916486    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916489    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916492    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916495    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916498    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916501    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916503    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916507    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916510    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916513    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916516    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916519    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916522    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.917015 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916526    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916529    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916532    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916535    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916537    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916540    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916544    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916546    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916549    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916552    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916555    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916558    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916560    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916563    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916565    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916568    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916571    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916573    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916577    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.917539 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916579    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916582    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916585    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916588    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916591    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916594    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916597    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916600    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916603    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916605    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916608    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916611    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916614    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916617    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916619    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916622    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916624    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916627    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916629    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.918028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916632    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916635    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916638    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916640    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916643    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916645    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916648    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916651    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916653    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916656    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916658    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916661    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916666    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916669    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916671    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916673    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916676    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916680    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916684    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.918514 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916687    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916690    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916693    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916696    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916699    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916701    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.916706    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917513    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917521    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917525    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917527    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917530    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917533    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917536    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917538    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917542    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917545    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917548    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917551    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.919007 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917554    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917557    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917559    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917562    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917564    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917567    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917571    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917573    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917576    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917578    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917581    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917584    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917586    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917589    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917592    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917595    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917597    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917600    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917603    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.919543 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917605    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917608    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917611    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917613    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917616    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917618    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917621    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917624    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917626    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917629    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917632    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917635    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917638    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917641    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917643    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917646    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917649    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917652    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917654    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.920019 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917657    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917661    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917663    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917666    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917669    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917671    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917674    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917677    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917679    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917682    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917684    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917687    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917690    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917694    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917698    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917703    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917706    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917709    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917711    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.920562 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917714    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917718    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917721    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917723    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917726    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917730    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917732    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917735    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917738    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917741    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917743    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917746    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917749    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917752    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917755    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917757    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917760    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917763    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917765    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917768    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.921050 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917771    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917773    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917777    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917781    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917784    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917786    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917789    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917792    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917794    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917797    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917799    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917802    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917805    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917807    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917810    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917812    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.917815    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918280    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918285    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.921571 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918288    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918291    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918294    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918298    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918301    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918303    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918306    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918309    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918312    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918315    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918318    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918320    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918323    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918325    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918328    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918331    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918333    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918336    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918338    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918341    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.922113 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918344    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918347    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918349    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918352    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918357    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918360    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918363    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918365    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918368    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918370    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918373    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918377    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918380    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918398    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918403    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918407    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918411    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918414    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918418    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.922751 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918422    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918424    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918427    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918430    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918433    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918435    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918439    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918442    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918444    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918447    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918450    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918453    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918456    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918458    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918461    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918463    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918466    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918470    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918473    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918476    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.923531 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918478    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918481    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918483    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918486    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918489    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918494    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918498    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918501    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918504    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918507    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918510    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918513    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918516    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918518    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918521    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918524    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918527    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918530    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918532    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.924099 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918535    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918538    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918540    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918543    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918546    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918548    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918551    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918554    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918558    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918562    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918566    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918576    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918580    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918583    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918585    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918588    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918591    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918595    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918598    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.924723 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918602    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918605    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918607    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918610    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918613    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918615    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.918618    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918713    4852 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918724    4852 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918730    4852 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918735    4852 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918740    4852 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918743    4852 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918747    4852 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918752    4852 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918755    4852 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918759    4852 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918763    4852 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918766    4852 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918769    4852 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918772    4852 flags.go:64] FLAG: --cgroup-root=""
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918775    4852 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918779    4852 flags.go:64] FLAG: --client-ca-file=""
Oct 30 20:38:11.925209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918783    4852 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918786    4852 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918790    4852 flags.go:64] FLAG: --cluster-domain=""
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918793    4852 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918796    4852 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918799    4852 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918803    4852 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918808    4852 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918811    4852 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918815    4852 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918819    4852 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918822    4852 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918825    4852 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918828    4852 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918832    4852 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918835    4852 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918839    4852 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918842    4852 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918845    4852 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918848    4852 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918852    4852 flags.go:64] FLAG: --enable-server="true"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918855    4852 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918859    4852 flags.go:64] FLAG: --event-burst="100"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918862    4852 flags.go:64] FLAG: --event-qps="50"
Oct 30 20:38:11.925803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918866    4852 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918869    4852 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918872    4852 flags.go:64] FLAG: --eviction-hard=""
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918877    4852 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918880    4852 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918884    4852 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918887    4852 flags.go:64] FLAG: --eviction-soft=""
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918890    4852 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918894    4852 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918897    4852 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918900    4852 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918903    4852 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918906    4852 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918909    4852 flags.go:64] FLAG: --feature-gates=""
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918913    4852 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918918    4852 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918921    4852 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918925    4852 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918928    4852 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918932    4852 flags.go:64] FLAG: --help="false"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918935    4852 flags.go:64] FLAG: --hostname-override="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918938    4852 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918941    4852 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 20:38:11.926409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918944    4852 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918947    4852 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918951    4852 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918954    4852 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918957    4852 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918959    4852 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918962    4852 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918965    4852 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918969    4852 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918971    4852 flags.go:64] FLAG: --kube-reserved=""
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918974    4852 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918977    4852 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918980    4852 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918983    4852 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918986    4852 flags.go:64] FLAG: --lock-file=""
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918989    4852 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918992    4852 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.918997    4852 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919003    4852 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919005    4852 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919009    4852 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919012    4852 flags.go:64] FLAG: --logging-format="text"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919015    4852 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919018    4852 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 20:38:11.926990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919023    4852 flags.go:64] FLAG: --manifest-url=""
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919026    4852 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919031    4852 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919034    4852 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919038    4852 flags.go:64] FLAG: --max-pods="110"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919041    4852 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919045    4852 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919048    4852 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919050    4852 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919053    4852 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919056    4852 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919059    4852 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919066    4852 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919069    4852 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919072    4852 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919076    4852 flags.go:64] FLAG: --pod-cidr=""
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919079    4852 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919082    4852 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919085    4852 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919088    4852 flags.go:64] FLAG: --port="10250"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919091    4852 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919094    4852 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-067cdabc38b0fe16b"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919097    4852 flags.go:64] FLAG: --qos-reserved=""
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919100    4852 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919103    4852 flags.go:64] FLAG: --register-node="true"
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919108    4852 flags.go:64] FLAG: --register-with-taints=""
Oct 30 20:38:11.927676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919112    4852 flags.go:64] FLAG: --registry-burst="10"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919115    4852 flags.go:64] FLAG: --registry-qps="5"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919118    4852 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919121    4852 flags.go:64] FLAG: --reserved-memory=""
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919125    4852 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919128    4852 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919132    4852 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919136    4852 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919138    4852 flags.go:64] FLAG: --runonce="false"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919141    4852 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919145    4852 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919148    4852 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919151    4852 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919154    4852 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919157    4852 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919161    4852 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919164    4852 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919167    4852 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919170    4852 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919173    4852 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919176    4852 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919179    4852 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919182    4852 flags.go:64] FLAG: --system-cgroups=""
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919185    4852 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919190    4852 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 20:38:11.928425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919193    4852 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919196    4852 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919199    4852 flags.go:64] FLAG: --tls-min-version=""
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919202    4852 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919205    4852 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919208    4852 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919212    4852 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919215    4852 flags.go:64] FLAG: --v="2"
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919219    4852 flags.go:64] FLAG: --version="false"
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919223    4852 flags.go:64] FLAG: --vmodule=""
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919226    4852 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919230    4852 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919406    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919413    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919416    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919420    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919424    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919428    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919432    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919435    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919439    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919441    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.929061 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919444    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919447    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919449    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919452    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919455    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919458    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919460    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919463    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919465    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919468    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919471    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919474    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919476    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919479    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919482    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919484    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919488    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919491    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919493    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919496    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.929657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919499    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919501    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919504    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919508    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919511    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919514    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919517    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919520    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919522    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919525    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919527    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919530    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919533    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919536    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919539    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919541    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919544    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919547    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919550    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.930145 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919553    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919555    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919558    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919561    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919564    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919566    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919569    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919571    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919574    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919578    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919581    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919584    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919586    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919589    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919592    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919594    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919598    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919600    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919603    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.930750 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919606    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919609    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919611    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919614    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919617    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919620    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919623    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919626    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919629    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919631    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919634    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919636    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919639    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919641    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919644    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919647    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919650    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919652    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919655    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919658    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.931294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919660    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919663    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919667    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919669    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919672    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919675    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919677    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919680    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919683    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919686    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919688    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919691    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919693    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919697    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919701    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919705    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919708    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.932001 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919711    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.919736    4852 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919775    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919779    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919782    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919785    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919787    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919790    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919793    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919795    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919798    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919801    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919803    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919806    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919808    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919811    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.932464 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919814    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919816    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919820    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919824    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919826    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919829    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919831    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919834    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919837    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919840    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919842    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919845    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919848    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919851    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919855    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919858    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919861    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919863    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919866    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919869    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.932861 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919872    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919875    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919879    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919883    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919885    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919888    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919891    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919893    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919896    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919899    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919901    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919904    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919906    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919909    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919913    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919916    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919918    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919921    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919924    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.933478 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919927    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919930    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919933    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919935    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919938    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919940    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919943    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919946    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919948    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919951    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919953    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919956    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919958    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919961    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919964    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919967    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919969    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919972    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919974    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919977    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.934034 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919979    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919982    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919984    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919987    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919990    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919992    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.919995    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920001    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920004    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920006    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920009    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920011    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920015    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920018    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920021    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920024    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920026    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920029    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920032    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.934721 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920035    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920037    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920040    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920043    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920045    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920048    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920050    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920053    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920056    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920059    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920061    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920065    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920068    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.920070    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.925516    4852 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.925533    4852 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.925557    4852 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.925561    4852 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925622    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.935642 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925627    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925631    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925634    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925637    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925639    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925642    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925645    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925648    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925651    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925653    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925656    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925659    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925662    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925665    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925668    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925671    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925673    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925676    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925680    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925683    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.936446 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925685    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925688    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925691    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925694    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925696    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925699    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925702    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925705    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925708    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925710    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925713    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925716    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925719    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925722    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925726    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925731    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925734    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925737    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925739    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925742    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.937327 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925745    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925747    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925750    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925753    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925756    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925759    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925762    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925764    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925767    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925770    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925773    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925776    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925779    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925782    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925785    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925787    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925790    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925793    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925796    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.938167 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925799    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925802    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925805    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925808    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925811    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925814    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925817    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925820    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925823    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925825    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925828    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925831    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925833    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925836    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925838    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925841    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925844    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925847    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925849    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925853    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.939020 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925856    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925859    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925862    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925865    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925868    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925870    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925873    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925876    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925879    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925881    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925884    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925887    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925890    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925893    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925897    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925901    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925904    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925907    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925910    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.940029 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925913    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925916    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925919    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925922    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925924    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925927    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.925929    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.925958    4852 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926097    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926101    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926104    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926107    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926110    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926113    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926116    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926119    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.941045 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926122    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926125    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926128    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926130    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926133    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926135    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926138    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926141    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926144    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926148    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926152    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926155    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926158    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926161    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926164    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926166    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926169    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926172    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926175    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.941863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926177    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926180    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926183    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926186    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926189    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926191    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926194    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926197    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926201    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926204    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926208    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926212    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926215    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926217    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926220    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926223    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926226    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926229    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926231    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926234    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.942737 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926236    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926239    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926242    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926244    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926247    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926250    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926252    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926255    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926257    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926260    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926263    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926265    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926268    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926271    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926273    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926276    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926279    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926282    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926284    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926287    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.943761 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926289    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926292    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926295    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926298    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926301    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926304    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926306    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926309    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926312    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926314    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926317    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926320    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926322    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926325    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926328    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926330    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926333    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926335    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926338    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.944657 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926340    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926343    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926346    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926348    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926351    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926354    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926356    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926359    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926362    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926364    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926367    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926369    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926372    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926375    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926378    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926380    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926397    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926401    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926405    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.945265 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926407    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.926431    4852 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926464    4852 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926467    4852 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926470    4852 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926473    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926476    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926479    4852 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926481    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926484    4852 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926487    4852 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926490    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926493    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926495    4852 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926498    4852 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926500    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 20:38:11.945799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926503    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926506    4852 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926509    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926512    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926515    4852 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926518    4852 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926520    4852 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926523    4852 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926526    4852 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926528    4852 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926531    4852 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926533    4852 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926536    4852 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926538    4852 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926541    4852 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926544    4852 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926550    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926552    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926555    4852 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 20:38:11.946193 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926558    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926561    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926563    4852 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926566    4852 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926575    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926578    4852 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926581    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926584    4852 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926586    4852 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926589    4852 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926592    4852 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926594    4852 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926597    4852 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926599    4852 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926602    4852 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926605    4852 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926608    4852 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926610    4852 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926613    4852 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926615    4852 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 20:38:11.946692 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926618    4852 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926621    4852 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926623    4852 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926626    4852 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926629    4852 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926631    4852 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926634    4852 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926637    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926639    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926642    4852 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926646    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926648    4852 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926651    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926654    4852 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926657    4852 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926659    4852 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926664    4852 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926667    4852 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926671    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 20:38:11.947221 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926674    4852 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926677    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926680    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926682    4852 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926685    4852 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926688    4852 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926690    4852 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926693    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926696    4852 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926698    4852 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926701    4852 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926704    4852 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926706    4852 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926709    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926715    4852 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926719    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926722    4852 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926725    4852 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926727    4852 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 20:38:11.948042 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926730    4852 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926733    4852 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926737    4852 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926739    4852 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926742    4852 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926745    4852 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926748    4852 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926751    4852 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926753    4852 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926756    4852 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926759    4852 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926761    4852 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926764    4852 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926766    4852 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:11.926769    4852 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.926849    4852 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.927841    4852 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.927900    4852 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.928074    4852 server.go:1014] "Starting client certificate rotation"
Oct 30 20:38:11.948887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.928214    4852 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.928261    4852 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 20:32:29 +0000 UTC" deadline="2028-06-20 00:41:18.745683794 +0000 UTC"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.928290    4852 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14356h3m6.817395927s"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.928513    4852 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.929515    4852 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.931374    4852 log.go:25] "Validated CRI v1 runtime API"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.933196    4852 log.go:25] "Validated CRI v1 image API"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.933910    4852 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 20:38:11.949808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.936471    4852 fs.go:135] Filesystem UUIDs: map[5b58bffc-db78-4e46-accc-3193eb4625c7:/dev/nvme0n1p4 69f45577-64fd-46ac-93fa-9c1ee9f6ab08:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 30 20:38:11.950199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.936503    4852 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/07a2407b256803d7f2dad3b166aa086262eb72ab522faa08e1c2e28a9aef9466/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/07a2407b256803d7f2dad3b166aa086262eb72ab522faa08e1c2e28a9aef9466/userdata/shm major:0 minor:71 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0e7445d7143a33f386eea8bf81a5da02f863bcf5e074627b7317e219ea5a90e6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0e7445d7143a33f386eea8bf81a5da02f863bcf5e074627b7317e219ea5a90e6/userdata/shm major:0 minor:217 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/10cf589376990dc2b8cf887f858274dc52495014b70934ce59850591c23d95f8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/10cf589376990dc2b8cf887f858274dc52495014b70934ce59850591c23d95f8/userdata/shm major:0 minor:276 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/133e69e6bd70139e6ef630e17fef5fb3a28a6eba0d54d3b3e8e5ebe048138a96/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/133e69e6bd70139e6ef630e17fef5fb3a28a6eba0d54d3b3e8e5ebe048138a96/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/13a39580dc8eda0b6b525d28c4572c5ca116594e91a392999d4e1cd70c972c50/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/13a39580dc8eda0b6b525d28c4572c5ca116594e91a392999d4e1cd70c972c50/userdata/shm major:0 minor:261 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/16e6f45a96b4924ff58c8a8fded165b6fb4071af4edbb69e9a7f8d19a707bf2a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/16e6f45a96b4924ff58c8a8fded165b6fb4071af4edbb69e9a7f8d19a707bf2a/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1b17de172d54ac318797511723de1c03cb3979b7502b42d8e3e67a8e08ca3df4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1b17de172d54ac318797511723de1c03cb3979b7502b42d8e3e67a8e08ca3df4/userdata/shm major:0 minor:108 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/26cc749a619792700ce36c27c8b81337558ca7170bbe8f913d63283a549027c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/26cc749a619792700ce36c27c8b81337558ca7170bbe8f913d63283a549027c5/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/54ee5af7ec3419b96c81de00beb9b61a97c7e9388cfabbe603724387e4a0ac97/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/54ee5af7ec3419b96c81de00beb9b61a97c7e9388cfabbe603724387e4a0ac97/userdata/shm major:0 minor:274 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/76d66980b12fab770bd9ac58acb250d5183cad1ab6ddd94ca1596472fdadec7b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/76d66980b12fab770bd9ac58acb250d5183cad1ab6ddd94ca1596472fdadec7b/userdata/shm major:0 minor:115 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82f947d66fcee3374b788f0d89f84f777eca2fef04a91f5d489cbe4ecb999117/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82f947d66fcee3374b788f0d89f84f777eca2fef04a91f5d489cbe4ecb999117/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/904b70662e4a98f7dd9ceeda0c56a881389cbf09cf1fb81b31aa0bd60dece6d1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/904b70662e4a98f7dd9ceeda0c56a881389cbf09cf1fb81b31aa0bd60dece6d1/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/be06a115b0611fba33b94f8e57f95e70203f8ccf3344f0a83b00279b89738ae0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/be06a115b0611fba33b94f8e57f95e70203f8ccf3344f0a83b00279b89738ae0/userdata/shm major:0 minor:221 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cde8e180e9ed2e6e35293fbf2716b4ab30b1935b98447327c5b8135811ee0a52/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cde8e180e9ed2e6e35293fbf2716b4ab30b1935b98447327c5b8135811ee0a52/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc0015f062976097d9cef8376e1aa48796a3346add7ae17c284845f69a59f688/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc0015f062976097d9cef8376e1aa48796a3346add7ae17c284845f69a59f688/userdata/shm major:0 minor:104 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00295160-ccc3-481f-ac98-7b9e15aa7c25/volumes/kubernetes.io~projected/kube-api-access-z5xp2:{mountpoint:/var/lib/kubelet/pods/00295160-ccc3-481f-ac98-7b9e15aa7c25/volumes/kubernetes.io~projected/kube-api-access-z5xp2 major:0 minor:95 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ca03c7b-5428-4123-980b-9bdbff49850f/volumes/kubernetes.io~projected/kube-api-access-5jggs:{mountpoint:/var/lib/kubelet/pods/0ca03c7b-5428-4123-980b-9bdbff49850f/volumes/kubernetes.io~projected/kube-api-access-5jggs major:0 minor:271 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14627328-5fa1-4c44-93a1-94b03f420bc7/volumes/kubernetes.io~projected/kube-api-access-wds6s:{mountpoint:/var/lib/kubelet/pods/14627328-5fa1-4c44-93a1-94b03f420bc7/volumes/kubernetes.io~projected/kube-api-access-wds6s major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14627328-5fa1-4c44-93a1-94b03f420bc7/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/14627328-5fa1-4c44-93a1-94b03f420bc7/volumes/kubernetes.io~secret/metrics-tls major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volumes/kubernetes.io~projected/kube-api-access-8zbf2:{mountpoint:/var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volumes/kubernetes.io~projected/kube-api-access-8zbf2 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:93 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:92 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~projected/kube-api-access-f65gx:{mountpoint:/var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~projected/kube-api-access-f65gx major:0 minor:94 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087/volumes/kubernetes.io~projected/kube-api-access-jk4kc:{mountpoint:/var/lib/kubelet/pods/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087/volumes/kubernetes.io~projected/kube-api-access-jk4kc major:0 minor:68 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5108533b-2d1b-488f-a03c-fd5beb4f34f5/volumes/kubernetes.io~projected/kube-api-access-5wb89:{mountpoint:/var/lib/kubelet/pods/5108533b-2d1b-488f-a03c-fd5beb4f34f5/volumes/kubernetes.io~projected/kube-api-access-5wb89 major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5108533b-2d1b-488f-a03c-fd5beb4f34f5/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/5108533b-2d1b-488f-a03c-fd5beb4f34f5/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~projected/kube-api-access-h6mhd:{mountpoint:/var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~projected/kube-api-access-h6mhd major:0 minor:114 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:113 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/754d8def-099b-4fdb-859c-0f318bf5fd38/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/754d8def-099b-4fdb-859c-0f318bf5fd38/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c7b0506-8e4a-43bd-b7a6-e58e9879d893/volumes/kubernetes.io~projected/kube-api-access-5rl4z:{mountpoint:/var/lib/kubelet/pods/9c7b0506-8e4a-43bd-b7a6-e58e9879d893/volumes/kubernetes.io~projected/kube-api-access-5rl4z major:0 minor:102 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a0b8f9f1-3e33-40b5-bcc2-743909a0295f/volumes/kubernetes.io~projected/kube-api-access-4kww9:{mountpoint:/var/lib/kubelet/pods/a0b8f9f1-3e33-40b5-bcc2-743909a0295f/volumes/kubernetes.io~projected/kube-api-access-4kww9 major:0 minor:103 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8ff15b4-7b0f-4e99-883c-05938faa37df/volumes/kubernetes.io~projected/kube-api-access-gdt9f:{mountpoint:/var/lib/kubelet/pods/a8ff15b4-7b0f-4e99-883c-05938faa37df/volumes/kubernetes.io~projected/kube-api-access-gdt9f major:0 minor:69 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d5a8e9f5-484c-4848-a012-21b8ec6ecb32/volumes/kubernetes.io~projected/kube-api-access-ls8xd:{mountpoint:/var/lib/kubelet/pods/d5a8e9f5-484c-4848-a012-21b8ec6ecb32/volumes/kubernetes.io~projected/kube-api-access-ls8xd major:0 minor:84 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d5a8e9f5-484c-4848-a012-21b8ec6ecb32/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/d5a8e9f5-484c-4848-a012-21b8ec6ecb32/volumes/kubernetes.io~secret/metrics-certs major:0 minor:260 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0d61b27-34af-40d5-8ab0-b06c28aa86e4/volumes/kubernetes.io~projected/kube-api-access-pcj5d:{mountpoint:/var/lib/kubelet/pods/f0d61b27-34af-40d5-8ab0-b06c28aa86e4/volumes/kubernetes.io~projected/kube-api-access-pcj5d major:0 minor:82 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f314cf90-719e-4d6d-8745-0ff46bbbf9f8/volumes/kubernetes.io~projected/kube-api-access-qklrf:{mountpoint:/var/lib/kubelet/pods/f314cf90-719e-4d6d-8745-0ff46bbbf9f8/volumes/kubernetes.io~projected/kube-api-access-qklrf major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f314cf90-719e-4d6d-8745-0ff46bbbf9f8/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/f314cf90-719e-4d6d-8745-0ff46bbbf9f8/volumes/kubernetes.io~secret/cert major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/volumes/kubernetes.io~secret/agent-certs major:0 minor:59 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/d962d8c523a4cda5e58ca88691c71d9dc7f59ec6fa2e5cd9fe9208a70d9df618/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/caeb59f3596d4be6b590b500b84125a83d88205077feefdf82c71b3770903472/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-110:{mountpoint:/var/lib/containers/storage/overlay/0d7b83814514a545fe77a5c717e51812a354579fdf5cd3e7d4f042a785ecb889/merged major:0 minor:110 fsType:overlay blockSize:0} overlay_0-117:{mountpoint:/var/lib/containers/storage/overlay/d732b914c689f6ea8fb1dc4678f9bd1fb3c68644370e4c83aae5dd6279a0b122/merged major:0 minor:117 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/04faa23a7c3c126dba4eae166cc59d3f98a8755eecfc50e84e102dab4c75aa4f/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-121:{mountpoint:/var/lib/containers/storage/overlay/e7dd82a6bf9b1a66270c663b14964ec7f7216358af353522974c71168c881adc/merged major:0 minor:121 fsType:overlay blockSize:0} overlay_0-126:{mountpoint:/var/lib/containers/storage/overlay/0420fcf9efcfad769852ff94cc275e011fcbfdd3f68af59cec6630e77ba818a1/merged major:0 minor:126 fsType:overlay blockSize:0} overlay_0-128:{mountpoint:/var/lib/containers/storage/overlay/4780fbebda8a5abdfa79aaaf31f3ce7ea32755f6d4f72a901078de3b1642ab59/merged major:0 minor:128 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/b90d1ee226091329eb1763f03f17cfd6bd13084d9871bc380d63044f37f57abf/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/92ca5ac771dafea82126fe380f50cc0f350b8baa04828299ec3f41e0739ca234/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/2faf536ad510a1ce25d0717b8215ce0dc0d4ac382984b12cc5a91ad5aaaf8f67/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/3105c4cef961fd125f8f06627066fb9f576c88031ad7742de890a955d533a3e4/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-156:{mountpoint:/var/lib/containers/storage/overlay/4840bc0c55eb619bc3b146ca4ce7a1ba6d2075a0882d53fc1d19c67ac689e47e/merged major:0 minor:156 fsType:overlay blockSize:0} overlay_0-164:{mountpoint:/var/lib/containers/storage/overlay/02197e009d8e70b759646d424ba7d573e4c9e107c70e2dc368d1a6cedd249276/merged major:0 minor:164 fsType:overlay blockSize:0} overlay_0-169:{mountpoint:/var/lib/containers/storage/overlay/62282d721e31e2fbd3de0dcb77c115dabe485e37e90205e394bc7bf9d496ae2a/merged major:0 minor:169 fsType:overlay blockSize:0} overlay_0-174:{mountpoint:/var/lib/containers/storage/overlay/cd14e8081d1159d7984b80645114c25230e645aa983bc70cfcd9e12364b45a9b/merged major:0 minor:174 fsType:overlay blockSize:0} overlay_0-179:{mountpoint:/var/lib/containers/storage/overlay/4dcf809c6ebdc6b35d8356d59cb04ca54a1dcedc30b9ca84c1b99cf5f0d25ea7/merged major:0 minor:179 fsType:overlay blockSize:0} overlay_0-184:{mountpoint:/var/lib/containers/storage/overlay/5d8cdd22689222b4213d147fa66ceec7e95772be25e00ee2fc5653e7c3bcdf91/merged major:0 minor:184 fsType:overlay blockSize:0} overlay_0-189:{mountpoint:/var/lib/containers/storage/overlay/2b912e089c4f6c3af3008968f6d15ef08d96467d837294201d72b979af41aaf6/merged major:0 minor:189 fsType:overlay blockSize:0} overlay_0-198:{mountpoint:/var/lib/containers/storage/overlay/97fb5c0ada33542fda5b8aa8b7ac9cb19343ee1f5897b0df9cff9b402beaa32c/merged major:0 minor:198 fsType:overlay blockSize:0} overlay_0-203:{mountpoint:/var/lib/containers/storage/overlay/d10629bc4525f0e2ab6bf355161c0337b154925b2bda407e8c4ed5e86c7a4d73/merged major:0 minor:203 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/375d83240dd184c446a5276cbac6e3e0bca22f6e405dd6ab50e7ad17f2f0a92c/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/b09aeb08fb5ac72a01ef386cf3144f232ee3cffd30ad73ddb10051c13b0f1881/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/21f9ebd0704e3ea725b1162f2057d1746004b5e95d08806af0b53789540056e3/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-229:{mountpoint:/var/lib/containers/storage/overlay/d4ab793449e8b2c79d7e77d491ab6ee7b6ba96f5cacc18b705ef39e1fff62d05/merged major:0 minor:229 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/575984ab5d69ef6820ef80de853bcdde3457fd230d15e68a2f270f22cb6aae3e/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/c9c10cb1c8a4c5327a34d0f89a7e19f306d8b863cb9f6957f574ed3b2550aa1d/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/c6c7aaf578bda81244330d92a923eead6324302dbd22153c2c923ffa0c3cb290/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/35555effef3fab1cff1faa13231f974be2d396089a7eeaddb31496099871e5a0/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/0829ca0d8a4579a238dcef411bfb5668264f6b8745d18fbff2bf7d0c0695032e/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/d160f4fe3fb2c0e891a4b00d287d023d22b99d505c7ef0833e0bc7e8affc92ce/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/aaf23cd9fb5664749acda18140074d3ce55c0616faef0106dbf63fd884b1a474/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/d7cde03baf5281dd286a2f6e57bdabf6840fc754c55edf313ada0a0d015cd1b6/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/48e4c81f9ba4e3e0789f8549719e6b511119118da62dddceadc8ac7818a68c3c/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/99286d852a931acf0dfbbd8b92ae2579acea9da3c65f2a0b76d9d946e16ca558/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/b12d242540eb8ec8ae3337224fa23d2caf2e82a95efe9a039b41b789bbcfec12/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/8bd6f0169d6245b8942aa949c779ac16eeec92bdbe89a5912bd139af2b30127d/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-295:{mountpoint:/var/lib/containers/storage/overlay/c5ff515087212dfae3e04a915675df4b4eefec005d584cfe4a76cbae28dac5c0/merged major:0 minor:295 fsType:overlay blockSize:0} overlay_0-45:{mountpoint:/var/lib/containers/storage/overlay/49459f87e05d74f0a4f9c2e1e314683c9312cdac33f90f5c2696de9819578571/merged major:0 minor:45 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/1e1dedd2b62e4ac282e07c34472cafd67aabdd4549b874c6e1ff24341f6853c8/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/c5afc91322a5f025b6351e6f7825926b228d6982e9533bd624dc2f4fb6d00ab7/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-55:{mountpoint:/var/lib/containers/storage/overlay/f4562b2ab1fd16f829b449aa1fd5b3afb13ea7478cfe79f055a3ae8efd983244/merged major:0 minor:55 fsType:overlay blockSize:0} overlay_0-57:{mountpoint:/var/lib/containers/storage/overlay/288ad38dc1df48d68623dc53fcb68ad6011f429ef2adbbb682582235e37a16e1/merged major:0 minor:57 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/1e881002851e0a4b7dbbf96c5cfe17778d42e6340532addf8d2aa2077e208dd3/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-73:{mountpoint:/var/lib/containers/storage/overlay/a4c12db34e2ac4523f367b0354b588a76be839da184653e81fba898b1c3eda15/merged major:0 minor:73 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/80ffc1d3dc0d6530ee5e1aeffca241e7eaa9c8fa8aad1864be6dbd8565f58f22/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/b8c944d2e875177b0f685d347e81abaf36279b39f9a946a99e5231c77ebd303a/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/fbcce04e31181de60e475082a55ab78bd44c67189a394abe67dd8855a90154c8/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/1795031e04dd10eb66aeff1e8b8f2988448643ed8db9e95d303beed069cd242c/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/d0d46dde04c4a2b0329ec189caeb3f24769879905a42c85f3ccfd9ec88ce6d16/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.945957    4852 manager.go:217] Machine: {Timestamp:2026-10-30 20:38:11.945497471 +0000 UTC m=+0.060231652 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100484 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec200d715fd40e77dd436ead884d3aad SystemUUID:ec200d71-5fd4-0e77-dd43-6ead884d3aad BootID:0289e18c-fa19-4ccf-ab6c-498e3a26f0db Filesystems:[{Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d5a8e9f5-484c-4848-a012-21b8ec6ecb32/volumes/kubernetes.io~projected/kube-api-access-ls8xd DeviceMajor:0 DeviceMinor:84 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-184 DeviceMajor:0 DeviceMinor:184 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/133e69e6bd70139e6ef630e17fef5fb3a28a6eba0d54d3b3e8e5ebe048138a96/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/16e6f45a96b4924ff58c8a8fded165b6fb4071af4edbb69e9a7f8d19a707bf2a/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:93 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-198 DeviceMajor:0 DeviceMinor:198 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f314cf90-719e-4d6d-8745-0ff46bbbf9f8/volumes/kubernetes.io~projected/kube-api-access-qklrf DeviceMajor:0 DeviceMinor:212 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/14627328-5fa1-4c44-93a1-94b03f420bc7/volumes/kubernetes.io~projected/kube-api-access-wds6s DeviceMajor:0 DeviceMinor:216 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/10cf589376990dc2b8cf887f858274dc52495014b70934ce59850591c23d95f8/userdata/shm DeviceMajor:0 DeviceMinor:276 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-57 DeviceMajor:0 DeviceMinor:57 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/00295160-ccc3-481f-ac98-7b9e15aa7c25/volumes/kubernetes.io~projected/kube-api-access-z5xp2 DeviceMajor:0 DeviceMinor:95 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-121 DeviceMajor:0 DeviceMinor:121 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-174 DeviceMajor:0 DeviceMinor:174 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/14627328-5fa1-4c44-93a1-94b03f420bc7/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:214 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/754d8def-099b-4fdb-859c-0f318bf5fd38/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:270 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f314cf90-719e-4d6d-8745-0ff46bbbf9f8/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:208 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5108533b-2d1b-488f-a03c-fd5beb4f34f5/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:213 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/be06a115b0611fba33b94f8e57f95e70203f8ccf3344f0a83b00279b89738ae0/userdata/shm DeviceMajor:0 DeviceMinor:221 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-45 DeviceMajor:0 DeviceMinor:45 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-229 DeviceMajor:0 DeviceMinor:229 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/54ee5af7ec3419b96c81de00beb9b61a97c7e9388cfabbe603724387e4a0ac97/userdata/shm DeviceMajor:0 DeviceMinor:274 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~projected/kube-api-access-f65gx DeviceMajor:0 DeviceMinor:94 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-189 DeviceMajor:0 DeviceMinor:189 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0e7445d7143a33f386eea8bf81a5da02f863bcf5e074627b7317e219ea5a90e6/userdata/shm DeviceMajor:0 DeviceMinor:217 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/13a39580dc8eda0b6b525d28c4572c5ca116594e91a392999d4e1cd70c972c50/userdata/shm DeviceMajor:0 DeviceMinor:261 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-128 DeviceMajor:0 DeviceMinor:128 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/07a2407b256803d7f2dad3b166aa086262eb72ab522faa08e1c2e28a9aef9466/userdata/shm DeviceMajor:0 DeviceMinor:71 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5108533b-2d1b-488f-a03c-fd5beb4f34f5/volumes/kubernetes.io~projected/kube-api-access-5wb89 DeviceMajor:0 DeviceMinor:215 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-295 DeviceMajor:0 DeviceMinor:295 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volumes/kubernetes.io~projected/kube-api-access-8zbf2 DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1b17de172d54ac318797511723de1c03cb3979b7502b42d8e3e67a8e08ca3df4/userdata/shm DeviceMajor:0 DeviceMinor:108 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-126 DeviceMajor:0 DeviceMinor:126 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-169 DeviceMajor:0 DeviceMinor:169 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/904b70662e4a98f7dd9ceeda0c56a881389cbf09cf1fb81b31aa0bd60dece6d1/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0ca03c7b-5428-4123-980b-9bdbff49850f/volumes/kubernetes.io~projected/kube-api-access-5jggs DeviceMajor:0 DeviceMinor:271 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/26cc749a619792700ce36c27c8b81337558ca7170bbe8f913d63283a549027c5/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/9c7b0506-8e4a-43bd-b7a6-e58e9879d893/volumes/kubernetes.io~projected/kube-api-access-5rl4z DeviceMajor:0 DeviceMinor:102 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-179 DeviceMajor:0 DeviceMinor:179 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087/volumes/kubernetes.io~projected/kube-api-access-jk4kc DeviceMajor:0 DeviceMinor:68 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-73 DeviceMajor:0 DeviceMinor:73 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82f947d66fcee3374b788f0d89f84f777eca2fef04a91f5d489cbe4ecb999117/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/76d66980b12fab770bd9ac58acb250d5183cad1ab6ddd94ca1596472fdadec7b/userdata/shm DeviceMajor:0 DeviceMinor:115 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d5a8e9f5-484c-4848-a012-21b8ec6ecb32/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:260 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc0015f062976097d9cef8376e1aa48796a3346add7ae17c284845f69a59f688/userdata/shm DeviceMajor:0 DeviceMinor:104 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-110 DeviceMajor:0 DeviceMinor:110 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/260a13a6-e92d-4243-991e-ba4639127adc/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a8ff15b4-7b0f-4e99-883c-05938faa37df/volumes/kubernetes.io~projected/kube-api-access-gdt9f DeviceMajor:0 DeviceMinor:69 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cde8e180e9ed2e6e35293fbf2716b4ab30b1935b98447327c5b8135811ee0a52/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/f0d61b27-34af-40d5-8ab0-b06c28aa86e4/volumes/kubernetes.io~projected/kube-api-access-pcj5d DeviceMajor:0 DeviceMinor:82 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a0b8f9f1-3e33-40b5-bcc2-743909a0295f/volumes/kubernetes.io~projected/kube-api-access-4kww9 DeviceMajor:0 DeviceMinor:103 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-55 DeviceMajor:0 DeviceMinor:55 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-117 DeviceMajor:0 DeviceMinor:117 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-203 DeviceMajor:0 DeviceMinor:203 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:92 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:113 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/70f4b787-e9c9-4e85-894d-27d370f8766d/volumes/kubernetes.io~projected/kube-api-access-h6mhd DeviceMajor:0 DeviceMinor:114 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-156 DeviceMajor:0 DeviceMinor:156 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-164 DeviceMajor:0 DeviceMinor:164 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0e7445d7143a33f MacAddress:e2:0f:22:7c:f3:6f Speed:10000 Mtu:8901} {Name:10cf589376990dc MacAddress:be:53:cc:3f:bf:68 Speed:10000 Mtu:8901} {Name:13a39580dc8eda0 MacAddress:1e:39:91:19:98:1c Speed:10000 Mtu:8901} {Name:54ee5af7ec3419b MacAddress:1a:c9:fc:22:9b:1e Speed:10000 Mtu:8901} {Name:904b70662e4a98f MacAddress:da:a8:8c:6e:f7:67 Speed:10000 Mtu:8901} {Name:be06a115b0611fb MacAddress:ee:31:10:7b:04:3e Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:dd:91:29:59 Speed:0 Mtu:9001} {Name:br-int MacAddress:a2:06:c2:0d:28:45 Speed:0 Mtu:8901} {Name:ens5 MacAddress:02:ff:dd:91:29:59 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:2e:e7:91:4b:fd:38 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:6e:f9:75:11:d6:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946245    4852 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946290    4852 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946591    4852 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946619    4852 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-178.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946769    4852 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946777    4852 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946787    4852 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946804    4852 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.946870    4852 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947031    4852 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947134    4852 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947166    4852 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947175    4852 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947185    4852 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947195    4852 kubelet.go:405] "Adding apiserver pod source"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947203    4852 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947815    4852 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.947824    4852 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:38:11.950494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.948358    4852 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950524    4852 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950537    4852 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950639    4852 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950826    4852 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950852    4852 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950946    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950957    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950963    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950968    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950974    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950980    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950986    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950992    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.950999    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.951004    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.951021    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.951031    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.951060    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 20:38:11.951560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.951066    4852 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 20:38:11.954254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.954242    4852 server.go:1263] "Started kubelet"
Oct 30 20:38:11.954485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.954308    4852 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 20:38:11.954485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.954340    4852 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 20:38:11.954485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.954458    4852 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 20:38:11.955726 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.954770    4852 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 20:38:11.954837 ip-10-0-135-178 systemd[1]: Started Kubernetes Kubelet.
Oct 30 20:38:11.957321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.957289    4852 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 20:38:11.959793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.959766    4852 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 20:38:11.959875 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.959792    4852 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 20:38:11.960411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.959909    4852 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:32:38 +0000 UTC" deadline="2028-03-29 03:36:49.634365442 +0000 UTC"
Oct 30 20:38:11.960411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.959929    4852 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12366h58m37.674439541s"
Oct 30 20:38:11.961095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.960795    4852 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 20:38:11.961095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.960809    4852 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 20:38:11.961095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.960912    4852 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 20:38:11.961320 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:11.961136    4852 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-178.us-east-2.compute.internal\" not found"
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.962064    4852 factory.go:55] Registering systemd factory
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.962085    4852 factory.go:223] Registration of the systemd container factory successfully
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.962981    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00295160-ccc3-481f-ac98-7b9e15aa7c25" volumeName="kubernetes.io/projected/00295160-ccc3-481f-ac98-7b9e15aa7c25-kube-api-access-z5xp2" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963012    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5108533b-2d1b-488f-a03c-fd5beb4f34f5" volumeName="kubernetes.io/configmap/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963026    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70f4b787-e9c9-4e85-894d-27d370f8766d" volumeName="kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963040    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5a8e9f5-484c-4848-a012-21b8ec6ecb32" volumeName="kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963052    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d61b27-34af-40d5-8ab0-b06c28aa86e4" volumeName="kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-daemon-config" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963063    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd3db454-59fb-46b2-90e9-3b7f4c11d5bb" volumeName="kubernetes.io/configmap/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-konnectivity-ca" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963070    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ca03c7b-5428-4123-980b-9bdbff49850f" volumeName="kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963078    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14627328-5fa1-4c44-93a1-94b03f420bc7" volumeName="kubernetes.io/empty-dir/14627328-5fa1-4c44-93a1-94b03f420bc7-tmp-dir" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963088    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260a13a6-e92d-4243-991e-ba4639127adc" volumeName="kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963096    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5108533b-2d1b-488f-a03c-fd5beb4f34f5" volumeName="kubernetes.io/empty-dir/5108533b-2d1b-488f-a03c-fd5beb4f34f5-data-volume" seLinuxMountContext=""
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963029    4852 factory.go:153] Registering CRI-O factory
Oct 30 20:38:11.963146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963121    4852 factory.go:223] Registration of the crio container factory successfully
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963174    4852 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963191    4852 factory.go:103] Registering Raw factory
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963205    4852 manager.go:1199] Started watching for new ooms in manager
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963106    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14627328-5fa1-4c44-93a1-94b03f420bc7" volumeName="kubernetes.io/secret/14627328-5fa1-4c44-93a1-94b03f420bc7-metrics-tls" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963444    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0" volumeName="kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-tmp" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963470    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f7516cb-f8ee-4e50-88ae-f2cd6cd40087" volumeName="kubernetes.io/configmap/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-serviceca" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963486    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70f4b787-e9c9-4e85-894d-27d370f8766d" volumeName="kubernetes.io/empty-dir/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-textfile" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963507    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70f4b787-e9c9-4e85-894d-27d370f8766d" volumeName="kubernetes.io/projected/70f4b787-e9c9-4e85-894d-27d370f8766d-kube-api-access-h6mhd" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963520    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c7b0506-8e4a-43bd-b7a6-e58e9879d893" volumeName="kubernetes.io/projected/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-kube-api-access-5rl4z" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963534    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260a13a6-e92d-4243-991e-ba4639127adc" volumeName="kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-env-overrides" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963548    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ff15b4-7b0f-4e99-883c-05938faa37df" volumeName="kubernetes.io/projected/a8ff15b4-7b0f-4e99-883c-05938faa37df-kube-api-access-gdt9f" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963561    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f314cf90-719e-4d6d-8745-0ff46bbbf9f8" volumeName="kubernetes.io/projected/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-kube-api-access-qklrf" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963574    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f314cf90-719e-4d6d-8745-0ff46bbbf9f8" volumeName="kubernetes.io/secret/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-cert" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963587    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f7516cb-f8ee-4e50-88ae-f2cd6cd40087" volumeName="kubernetes.io/projected/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-kube-api-access-jk4kc" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963601    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5108533b-2d1b-488f-a03c-fd5beb4f34f5" volumeName="kubernetes.io/secret/5108533b-2d1b-488f-a03c-fd5beb4f34f5-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963614    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70f4b787-e9c9-4e85-894d-27d370f8766d" volumeName="kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-tls" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963627    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70f4b787-e9c9-4e85-894d-27d370f8766d" volumeName="kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963641    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" volumeName="kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963654    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d61b27-34af-40d5-8ab0-b06c28aa86e4" volumeName="kubernetes.io/projected/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-kube-api-access-pcj5d" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963667    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260a13a6-e92d-4243-991e-ba4639127adc" volumeName="kubernetes.io/projected/260a13a6-e92d-4243-991e-ba4639127adc-kube-api-access-8zbf2" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963680    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" volumeName="kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-binary-copy" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963698    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0" volumeName="kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-tuned" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963711    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70f4b787-e9c9-4e85-894d-27d370f8766d" volumeName="kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-metrics-client-ca" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963724    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="754d8def-099b-4fdb-859c-0f318bf5fd38" volumeName="kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963737    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" volumeName="kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963750    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" volumeName="kubernetes.io/projected/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-kube-api-access-4kww9" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963764    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d5a8e9f5-484c-4848-a012-21b8ec6ecb32" volumeName="kubernetes.io/projected/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-kube-api-access-ls8xd" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963777    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd3db454-59fb-46b2-90e9-3b7f4c11d5bb" volumeName="kubernetes.io/secret/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-agent-certs" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963792    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14627328-5fa1-4c44-93a1-94b03f420bc7" volumeName="kubernetes.io/configmap/14627328-5fa1-4c44-93a1-94b03f420bc7-config-volume" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963804    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14627328-5fa1-4c44-93a1-94b03f420bc7" volumeName="kubernetes.io/projected/14627328-5fa1-4c44-93a1-94b03f420bc7-kube-api-access-wds6s" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963817    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260a13a6-e92d-4243-991e-ba4639127adc" volumeName="kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-config" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963832    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260a13a6-e92d-4243-991e-ba4639127adc" volumeName="kubernetes.io/secret/260a13a6-e92d-4243-991e-ba4639127adc-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963847    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0" volumeName="kubernetes.io/projected/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-kube-api-access-f65gx" seLinuxMountContext=""
Oct 30 20:38:11.963852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963861    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5108533b-2d1b-488f-a03c-fd5beb4f34f5" volumeName="kubernetes.io/projected/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-api-access-5wb89" seLinuxMountContext=""
Oct 30 20:38:11.965479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963891    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c7b0506-8e4a-43bd-b7a6-e58e9879d893" volumeName="kubernetes.io/empty-dir/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-tmp-dir" seLinuxMountContext=""
Oct 30 20:38:11.965479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963936    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d61b27-34af-40d5-8ab0-b06c28aa86e4" volumeName="kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cni-binary-copy" seLinuxMountContext=""
Oct 30 20:38:11.965479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963951    4852 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00295160-ccc3-481f-ac98-7b9e15aa7c25" volumeName="kubernetes.io/configmap/00295160-ccc3-481f-ac98-7b9e15aa7c25-iptables-alerter-script" seLinuxMountContext=""
Oct 30 20:38:11.965479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963964    4852 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 20:38:11.965479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963972    4852 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 20:38:11.965479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.963591    4852 manager.go:319] Starting recovery of all containers
Oct 30 20:38:11.975400 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.975362    4852 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:38:11.976035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.976011    4852 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 20:38:11.976836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.976816    4852 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:38:11.979260 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.979219    4852 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:38:11.995037 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.995002    4852 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 20:38:11.996460 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.996435    4852 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 20:38:11.996460 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.996453    4852 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 20:38:11.996634 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:11.996472    4852 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 20:38:11.996634 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:11.996509    4852 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 20:38:12.000665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.000644    4852 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 20:38:12.010907 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.010874    4852 generic.go:356] "Generic (PLEG): container finished" podID="70f4b787-e9c9-4e85-894d-27d370f8766d" containerID="60eae345a6153e47be7a77dcb24c9c282a2d4af9744389a065f12a97941e09da" exitCode=0
Oct 30 20:38:12.021841 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.021809    4852 generic.go:356] "Generic (PLEG): container finished" podID="457f0a554379acbff3f8f5b8e1c5a53f" containerID="4414e9acc09dbc9aefe8c3ef2c6cc77b9cadb27e4e2dc7d807871965d14b9a0c" exitCode=0
Oct 30 20:38:12.034452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.034420    4852 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="32c600acbade84541cd27bd718393442e516dff63b2c510dc184cc605e331b40" exitCode=0
Oct 30 20:38:12.034452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.034447    4852 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="8d339f08ef793c0d03283994a7f62ff7f2d045665555edc3f8123a360ee9edaa" exitCode=0
Oct 30 20:38:12.034644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.034458    4852 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="2f951ba6b578551ff6e991b6c8baad203619db7f82ac2e1b92f01983a917f64c" exitCode=0
Oct 30 20:38:12.034644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.034466    4852 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="2202412eae9a7f879c07f99c821823fca95e81b2bfe7a698b02cb0d28e43bc3f" exitCode=0
Oct 30 20:38:12.034644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.034472    4852 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="a3739d963adf3b79f87d3a8fc59ffafa3f8fe84df4858227fa054ab0b8bfb7eb" exitCode=0
Oct 30 20:38:12.034644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.034479    4852 generic.go:356] "Generic (PLEG): container finished" podID="a0b8f9f1-3e33-40b5-bcc2-743909a0295f" containerID="967a1dedee09acbcdf8d02323de68e24a22c8676a1ce123744d8c70e0127265b" exitCode=0
Oct 30 20:38:12.044234 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.044215    4852 manager.go:324] Recovery completed
Oct 30 20:38:12.061722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061698    4852 cpu_manager.go:226] "Starting" policy="none"
Oct 30 20:38:12.061722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061717    4852 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 20:38:12.061895 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061737    4852 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 20:38:12.061928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061895    4852 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 20:38:12.061928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061910    4852 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 20:38:12.061996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061930    4852 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 20:38:12.061996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061938    4852 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 20:38:12.061996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061946    4852 policy_none.go:50] "Start"
Oct 30 20:38:12.061996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061958    4852 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 20:38:12.061996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.061970    4852 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 20:38:12.062141 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.062100    4852 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 20:38:12.062141 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.062107    4852 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 20:38:12.062141 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.062112    4852 policy_none.go:44] "Start"
Oct 30 20:38:12.066280 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.066245    4852 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 20:38:12.068923 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.068908    4852 manager.go:341] "Starting Device Plugin manager"
Oct 30 20:38:12.069422 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:12.068938    4852 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 20:38:12.069508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.069424    4852 server.go:87] "Starting device plugin registration server"
Oct 30 20:38:12.069741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.069727    4852 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 20:38:12.069822 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.069742    4852 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 20:38:12.069881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.069842    4852 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 20:38:12.070007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.069966    4852 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 20:38:12.070076 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.070019    4852 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 20:38:12.070076 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.070027    4852 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 20:38:12.070160 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.070058    4852 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T20:38:12.070005784Z","UUID":"b1cc7c74-4667-45c9-a922-88b0363c7823","Handler":null,"Name":"","Endpoint":""}
Oct 30 20:38:12.071331 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:12.071315    4852 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 20:38:12.071331 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.071330    4852 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 20:38:12.071586 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.071349    4852 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 20:38:12.096746 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.096644    4852 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"]
Oct 30 20:38:12.097334 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.097293    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"3d0d587662eb50ad2820bb8aa4c2c1ec","Type":"ContainerStarted","Data":"9cd6f1b6355cc0ef5dc6b091f8b2ed220b73dfb9de6db24cb55082dc14212439"}
Oct 30 20:38:12.097404 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.097349    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"3d0d587662eb50ad2820bb8aa4c2c1ec","Type":"ContainerStarted","Data":"985db4a209af77481c12810a64eee366e89a99c4e22503cddcf84419f1c6ec81"}
Oct 30 20:38:12.097404 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.097370    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerStarted","Data":"a3212373aeca09679f1af89b0027024ff394fe48f4b3bb46ab1db5922f3e51be"}
Oct 30 20:38:12.097534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.097415    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerDied","Data":"4414e9acc09dbc9aefe8c3ef2c6cc77b9cadb27e4e2dc7d807871965d14b9a0c"}
Oct 30 20:38:12.097534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.097432    4852 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.097534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.097436    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal" event={"ID":"457f0a554379acbff3f8f5b8e1c5a53f","Type":"ContainerStarted","Data":"8e48b7f605a9dd4f0479d072e075ccaeb57d275f89dc81a7f1885bad6c919c0c"}
Oct 30 20:38:12.097534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.097486    4852 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.112124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.112101    4852 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:38:12.112170 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:12.112146    4852 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.112873 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.112856    4852 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:38:12.112912 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:12.112895    4852 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.170141 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.170102    4852 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 20:38:12.171352 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.171338    4852 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 20:38:12.171425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.171366    4852 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 20:38:12.171425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.171381    4852 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-178.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 20:38:12.171497 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.171438    4852 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.176626 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.176608    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.176676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.176634    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.176676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.176655    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d0d587662eb50ad2820bb8aa4c2c1ec-config\") pod \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"3d0d587662eb50ad2820bb8aa4c2c1ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.188339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.188317    4852 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.188463 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.188404    4852 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.277597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.277546    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.277597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.277603    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.277839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.277620    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d0d587662eb50ad2820bb8aa4c2c1ec-config\") pod \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"3d0d587662eb50ad2820bb8aa4c2c1ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.277839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.277639    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.277839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.277654    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d0d587662eb50ad2820bb8aa4c2c1ec-config\") pod \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"3d0d587662eb50ad2820bb8aa4c2c1ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.277839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.277658    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/457f0a554379acbff3f8f5b8e1c5a53f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" (UID: \"457f0a554379acbff3f8f5b8e1c5a53f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:12.948159 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.948128    4852 apiserver.go:52] "Watching apiserver"
Oct 30 20:38:12.957017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.956994    4852 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 20:38:12.957452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.957433    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-2rrmv","openshift-monitoring/node-exporter-qfl7v","openshift-ovn-kubernetes/ovnkube-node-825zz","openshift-ingress-canary/ingress-canary-mrq2q","kube-system/global-pull-secret-syncer-tck7z","openshift-insights/insights-runtime-extractor-rjlsv","kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal","openshift-image-registry/node-ca-29897","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal","openshift-multus/network-metrics-daemon-p52s5","kube-system/konnectivity-agent-bsg94","openshift-multus/multus-f7tx2","openshift-network-diagnostics/network-check-target-lhg4p","openshift-dns/node-resolver-g9hwf","openshift-multus/multus-additional-cni-plugins-qhl7c","openshift-dns/dns-default-ck9ws","openshift-network-operator/iptables-alerter-52cfv"]
Oct 30 20:38:12.964947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.964910    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.965092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965049    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 20:38:12.965142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965082    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.965142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965113    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 20:38:12.965243 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965164    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 20:38:12.965243 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965203    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.965371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965320    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.965452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965409    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.965452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965436    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.965554 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965515    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.966029 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966006    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.966132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966101    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.966252 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966196    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 20:38:12.966304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966293    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.966378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966369    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-82bc7\""
Oct 30 20:38:12.966551 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966416    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.966624 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966587    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bcgjk\""
Oct 30 20:38:12.966699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966628    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 20:38:12.966699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966666    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 20:38:12.966800 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.965322    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 20:38:12.967105 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.967073    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-4pvnz\""
Oct 30 20:38:12.967237 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.967197    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-chh5h\""
Oct 30 20:38:12.967321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.966199    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.967374 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.967316    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x97dd\""
Oct 30 20:38:12.967678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.967660    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.967806 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.967791    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l4rxj\""
Oct 30 20:38:12.968378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968357    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-jrl7k\""
Oct 30 20:38:12.968489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968429    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 20:38:12.968489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968448    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 20:38:12.968489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968464    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 20:38:12.968489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968485    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 20:38:12.968489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968494    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2hx48\""
Oct 30 20:38:12.968489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968495    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.968889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968431    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 20:38:12.968889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968546    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 20:38:12.968889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968582    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 20:38:12.968889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968601    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-6lk59\""
Oct 30 20:38:12.968889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.968698    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-7lq4m\""
Oct 30 20:38:12.969164 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.969068    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 20:38:12.969164 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.969129    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.969346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.969328    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zljsp\""
Oct 30 20:38:12.969464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.969410    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.969464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.969427    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fxgqk\""
Oct 30 20:38:12.972850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.972834    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 20:38:12.973699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.973681    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 20:38:12.973699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.973682    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.974350 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.974322    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 20:38:12.974415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.974372    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 20:38:12.974491 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.974477    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 20:38:12.974997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.974984    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.975035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975008    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.975199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975184    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 20:38:12.975398 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975370    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 20:38:12.975457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975405    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rsx8b\""
Oct 30 20:38:12.975687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975674    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.975900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975888    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5c74x\""
Oct 30 20:38:12.975987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975918    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 20:38:12.975987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975922    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-69hq7\""
Oct 30 20:38:12.975987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975930    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 20:38:12.975987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975969    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 20:38:12.975987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.975974    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 20:38:12.976151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.976036    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qtwx7\""
Oct 30 20:38:12.979930 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.979912    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-system-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.979937    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.979954    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-kubelet\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.979970    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.979984    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-node-log\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980018    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.980134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980062    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcj5d\" (UniqueName: \"kubernetes.io/projected/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-kube-api-access-pcj5d\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980086    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-conf\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.980134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980104    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdt9f\" (UniqueName: \"kubernetes.io/projected/a8ff15b4-7b0f-4e99-883c-05938faa37df-kube-api-access-gdt9f\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.980134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980120    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-systemd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980135    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-k8s-cni-cncf-io\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980187    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-config\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980259    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zbf2\" (UniqueName: \"kubernetes.io/projected/260a13a6-e92d-4243-991e-ba4639127adc-kube-api-access-8zbf2\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980265    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.980505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980426    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-hosts-file\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:38:12.980505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980470    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xp2\" (UniqueName: \"kubernetes.io/projected/00295160-ccc3-481f-ac98-7b9e15aa7c25-kube-api-access-z5xp2\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:38:12.980505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980496    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-systemd\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.980649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980514    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-lib-modules\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.980649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980537    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-bin\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980582    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-config\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980597    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-system-cni-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.980649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980630    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cni-binary-copy\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980687    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk4kc\" (UniqueName: \"kubernetes.io/projected/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-kube-api-access-jk4kc\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:38:12.980814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980760    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-sys\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.980814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980797    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-tmp\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.980922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980818    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-script-lib\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.980922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980852    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-conf-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980866    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-tmp\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.980922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980867    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-daemon-config\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980885    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cni-binary-copy\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.980922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980905    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-run\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.981134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.980997    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-var-lib-kubelet\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.981134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981023    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-ovnkube-script-lib\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981026    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-slash\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981028    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-daemon-config\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.981134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981062    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-host\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:38:12.981134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981102    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.981134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981120    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-netns\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981149    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/260a13a6-e92d-4243-991e-ba4639127adc-ovn-node-metrics-cert\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981177    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kww9\" (UniqueName: \"kubernetes.io/projected/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-kube-api-access-4kww9\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.981349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981213    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-var-lib-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981301    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981338    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.981572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981408    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-multus\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.981572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981434    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/260a13a6-e92d-4243-991e-ba4639127adc-ovn-node-metrics-cert\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981485    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/00295160-ccc3-481f-ac98-7b9e15aa7c25-iptables-alerter-script\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:38:12.981572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981543    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cni-binary-copy\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.981756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981616    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-systemd-units\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981642    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-log-socket\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.981756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981666    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-tmp-dir\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:38:12.981756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981697    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-bin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.981756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981735    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-tmp-dir\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:38:12.981984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981769    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-hostroot\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.981984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981777    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/00295160-ccc3-481f-ac98-7b9e15aa7c25-iptables-alerter-script\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:38:12.981984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981806    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-serviceca\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:38:12.981984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981880    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-host\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.981984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981912    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-socket-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.981984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981929    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rl4z\" (UniqueName: \"kubernetes.io/projected/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-kube-api-access-5rl4z\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:38:12.981984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.981949    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-netns\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982003    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysconfig\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982029    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982056    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cnibin\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982085    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-device-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982097    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-serviceca\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982121    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cnibin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982163    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982223    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-tuned\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982245    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-etc-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.982269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982261    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-env-overrides\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982277    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-etc-kubernetes\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982307    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982370    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-kubelet\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982425    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-tuned\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982441    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/260a13a6-e92d-4243-991e-ba4639127adc-env-overrides\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982451    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-ovn\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982481    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-socket-dir-parent\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982500    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls8xd\" (UniqueName: \"kubernetes.io/projected/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-kube-api-access-ls8xd\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982519    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-registration-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982537    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982566    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-metrics-certs\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982574    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982613    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982734    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-multus-certs\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982776    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982819    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/754d8def-099b-4fdb-859c-0f318bf5fd38-original-pull-secret\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982900    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00295160-ccc3-481f-ac98-7b9e15aa7c25-host-slash\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982918    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-kubernetes\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982932    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-sys-fs\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.982999    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-dbus\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.983016    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-modprobe-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.983038    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f65gx\" (UniqueName: \"kubernetes.io/projected/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-kube-api-access-f65gx\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.983070    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-netd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.983098    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-kubelet-config\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:12.983238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.983123    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.983589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.983263    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.987700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.987684    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcj5d\" (UniqueName: \"kubernetes.io/projected/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-kube-api-access-pcj5d\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:12.988026 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.988005    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xp2\" (UniqueName: \"kubernetes.io/projected/00295160-ccc3-481f-ac98-7b9e15aa7c25-kube-api-access-z5xp2\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:38:12.988337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.988319    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk4kc\" (UniqueName: \"kubernetes.io/projected/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-kube-api-access-jk4kc\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:38:12.988456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.988435    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zbf2\" (UniqueName: \"kubernetes.io/projected/260a13a6-e92d-4243-991e-ba4639127adc-kube-api-access-8zbf2\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:12.988515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.988487    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdt9f\" (UniqueName: \"kubernetes.io/projected/a8ff15b4-7b0f-4e99-883c-05938faa37df-kube-api-access-gdt9f\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:12.988975 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.988954    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jggs\" (UniqueName: \"kubernetes.io/projected/0ca03c7b-5428-4123-980b-9bdbff49850f-kube-api-access-5jggs\") pod \"network-check-target-lhg4p\" (UID: \"0ca03c7b-5428-4123-980b-9bdbff49850f\") " pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:12.989117 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.989104    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f65gx\" (UniqueName: \"kubernetes.io/projected/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-kube-api-access-f65gx\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:12.989158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.989138    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rl4z\" (UniqueName: \"kubernetes.io/projected/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-kube-api-access-5rl4z\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:38:12.989786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.989769    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kww9\" (UniqueName: \"kubernetes.io/projected/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-kube-api-access-4kww9\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:12.989847 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:12.989830    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls8xd\" (UniqueName: \"kubernetes.io/projected/d5a8e9f5-484c-4848-a012-21b8ec6ecb32-kube-api-access-ls8xd\") pod \"network-metrics-daemon-p52s5\" (UID: \"d5a8e9f5-484c-4848-a012-21b8ec6ecb32\") " pod="openshift-multus/network-metrics-daemon-p52s5"
Oct 30 20:38:13.059867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.059836    4852 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:13.060031 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.059972    4852 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:13.061396 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.061369    4852 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 20:38:13.070005 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.069966    4852 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:38:13.070088 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:13.070026    4852 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:13.071559 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.071543    4852 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 20:38:13.071600 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:13.071578    4852 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-178.us-east-2.compute.internal"
Oct 30 20:38:13.083933 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.083911    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-wtmp\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.084042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.083941    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-socket-dir-parent\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.084042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.083958    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-registration-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.084042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.083986    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084030    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084078    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qklrf\" (UniqueName: \"kubernetes.io/projected/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-kube-api-access-qklrf\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084095    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-root\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084107    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-socket-dir-parent\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084047    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084109    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084110    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-multus-certs\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084146    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-multus-certs\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084147    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00295160-ccc3-481f-ac98-7b9e15aa7c25-host-slash\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:38:13.084188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084173    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/00295160-ccc3-481f-ac98-7b9e15aa7c25-host-slash\") pod \"iptables-alerter-52cfv\" (UID: \"00295160-ccc3-481f-ac98-7b9e15aa7c25\") " pod="openshift-network-operator/iptables-alerter-52cfv"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084195    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-kubernetes\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084220    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-sys-fs\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084246    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-dbus\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084261    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-kubernetes\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084271    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-textfile\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084294    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-sys-fs\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084313    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-registration-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084328    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-dbus\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084332    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084361    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-textfile\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084374    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-modprobe-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084427    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-netd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084451    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-kubelet-config\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084482    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-modprobe-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084477    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-konnectivity-ca\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084527    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-netd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084530    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/754d8def-099b-4fdb-859c-0f318bf5fd38-kubelet-config\") pod \"global-pull-secret-syncer-tck7z\" (UID: \"754d8def-099b-4fdb-859c-0f318bf5fd38\") " pod="kube-system/global-pull-secret-syncer-tck7z"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084540    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wds6s\" (UniqueName: \"kubernetes.io/projected/14627328-5fa1-4c44-93a1-94b03f420bc7-kube-api-access-wds6s\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084565    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-system-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084581    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084596    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-kubelet\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.084603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084618    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084644    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-node-log\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084648    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-kubelet\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084650    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084671    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wb89\" (UniqueName: \"kubernetes.io/projected/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-api-access-5wb89\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084685    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084693    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-konnectivity-ca\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084693    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-accelerators-collector-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084746    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-conf\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084749    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-system-cni-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084775    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-systemd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084802    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-systemd\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084774    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-node-log\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084815    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-k8s-cni-cncf-io\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084834    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-hosts-file\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084850    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-systemd\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084874    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-lib-modules\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084889    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-conf\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084934    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-systemd\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084953    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-lib-modules\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084985    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c7b0506-8e4a-43bd-b7a6-e58e9879d893-hosts-file\") pod \"node-resolver-g9hwf\" (UID: \"9c7b0506-8e4a-43bd-b7a6-e58e9879d893\") " pod="openshift-dns/node-resolver-g9hwf"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084984    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-k8s-cni-cncf-io\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.084990    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-bin\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085026    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-system-cni-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085041    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-cni-bin\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085069    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085084    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-system-cni-dir\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085123    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-sys\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085150    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-conf-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085190    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-sys\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085190    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-run\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085221    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-var-lib-kubelet\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085237    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-slash\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085239    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-run\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085262    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-tls\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085277    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-multus-conf-dir\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085292    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-slash\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085303    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14627328-5fa1-4c44-93a1-94b03f420bc7-config-volume\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085318    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085324    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-host\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085346    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f7516cb-f8ee-4e50-88ae-f2cd6cd40087-host\") pod \"node-ca-29897\" (UID: \"4f7516cb-f8ee-4e50-88ae-f2cd6cd40087\") " pod="openshift-image-registry/node-ca-29897"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085373    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085419    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-netns\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085438    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085450    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-run-netns\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085452    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6mhd\" (UniqueName: \"kubernetes.io/projected/70f4b787-e9c9-4e85-894d-27d370f8766d-kube-api-access-h6mhd\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085479    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5108533b-2d1b-488f-a03c-fd5beb4f34f5-crio-socket\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085493    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/14627328-5fa1-4c44-93a1-94b03f420bc7-config-volume\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085499    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-tls\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085510    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5108533b-2d1b-488f-a03c-fd5beb4f34f5-data-volume\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085550    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-var-lib-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.085545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085570    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085594    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-sys\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085610    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-multus\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085612    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-var-lib-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085620    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5108533b-2d1b-488f-a03c-fd5beb4f34f5-data-volume\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085625    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-systemd-units\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085647    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085677    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-multus\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085681    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-log-socket\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085694    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-var-lib-kubelet\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085698    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-systemd-units\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085706    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-log-socket\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085738    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14627328-5fa1-4c44-93a1-94b03f420bc7-tmp-dir\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085788    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14627328-5fa1-4c44-93a1-94b03f420bc7-tmp-dir\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085793    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-bin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085809    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-hostroot\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085815    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-var-lib-cni-bin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085838    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-host\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085854    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-socket-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085868    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-cert\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085889    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085930    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-socket-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085940    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-hostroot\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.085949    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-host\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086022    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-netns\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086049    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysconfig\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086065    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-host-run-netns\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086074    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086099    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysconfig\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086075    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086105    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cnibin\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086123    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086133    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-cert\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086135    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5108533b-2d1b-488f-a03c-fd5beb4f34f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086138    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0b8f9f1-3e33-40b5-bcc2-743909a0295f-cnibin\") pod \"multus-additional-cni-plugins-qhl7c\" (UID: \"a0b8f9f1-3e33-40b5-bcc2-743909a0295f\") " pod="openshift-multus/multus-additional-cni-plugins-qhl7c"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086280    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5108533b-2d1b-488f-a03c-fd5beb4f34f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086425    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-agent-certs\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086178    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd3db454-59fb-46b2-90e9-3b7f4c11d5bb-agent-certs\") pod \"konnectivity-agent-bsg94\" (UID: \"fd3db454-59fb-46b2-90e9-3b7f4c11d5bb\") " pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086590    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14627328-5fa1-4c44-93a1-94b03f420bc7-metrics-tls\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086606    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-device-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086619    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cnibin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086633    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086656    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-etc-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086671    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-metrics-client-ca\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086680    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a8ff15b4-7b0f-4e99-883c-05938faa37df-device-dir\") pod \"aws-ebs-csi-driver-node-4gjbt\" (UID: \"a8ff15b4-7b0f-4e99-883c-05938faa37df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4gjbt"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086685    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-etc-kubernetes\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086718    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-etc-kubernetes\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086735    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-etc-openvswitch\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086744    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f0d61b27-34af-40d5-8ab0-b06c28aa86e4-cnibin\") pod \"multus-f7tx2\" (UID: \"f0d61b27-34af-40d5-8ab0-b06c28aa86e4\") " pod="openshift-multus/multus-f7tx2"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086928    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-kubelet\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086769    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-host-kubelet\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.087173    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/14627328-5fa1-4c44-93a1-94b03f420bc7-metrics-tls\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.086770    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0-etc-sysctl-d\") pod \"tuned-2rrmv\" (UID: \"3c1a0ec9-f2ec-4e98-b8a0-a6f49a9e52d0\") " pod="openshift-cluster-node-tuning-operator/tuned-2rrmv"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.087200    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70f4b787-e9c9-4e85-894d-27d370f8766d-metrics-client-ca\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.087197    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-ovn\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.087375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.087243    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/260a13a6-e92d-4243-991e-ba4639127adc-run-ovn\") pod \"ovnkube-node-825zz\" (UID: \"260a13a6-e92d-4243-991e-ba4639127adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:13.095454 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.095426    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wds6s\" (UniqueName: \"kubernetes.io/projected/14627328-5fa1-4c44-93a1-94b03f420bc7-kube-api-access-wds6s\") pod \"dns-default-ck9ws\" (UID: \"14627328-5fa1-4c44-93a1-94b03f420bc7\") " pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:13.095569 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.095426    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wb89\" (UniqueName: \"kubernetes.io/projected/5108533b-2d1b-488f-a03c-fd5beb4f34f5-kube-api-access-5wb89\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.095789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.095768    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6mhd\" (UniqueName: \"kubernetes.io/projected/70f4b787-e9c9-4e85-894d-27d370f8766d-kube-api-access-h6mhd\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.095946 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.095929    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qklrf\" (UniqueName: \"kubernetes.io/projected/f314cf90-719e-4d6d-8745-0ff46bbbf9f8-kube-api-access-qklrf\") pod \"ingress-canary-mrq2q\" (UID: \"f314cf90-719e-4d6d-8745-0ff46bbbf9f8\") " pod="openshift-ingress-canary/ingress-canary-mrq2q"
Oct 30 20:38:13.187599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187567    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5108533b-2d1b-488f-a03c-fd5beb4f34f5-crio-socket\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.187599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187609    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-sys\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.187827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187650    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-wtmp\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.187827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187675    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-root\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.187827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187729    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5108533b-2d1b-488f-a03c-fd5beb4f34f5-crio-socket\") pod \"insights-runtime-extractor-rjlsv\" (UID: \"5108533b-2d1b-488f-a03c-fd5beb4f34f5\") " pod="openshift-insights/insights-runtime-extractor-rjlsv"
Oct 30 20:38:13.187827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187751    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-root\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.187827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187730    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-sys\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:13.187971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:13.187826    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/70f4b787-e9c9-4e85-894d-27d370f8766d-node-exporter-wtmp\") pod \"node-exporter-qfl7v\" (UID: \"70f4b787-e9c9-4e85-894d-27d370f8766d\") " pod="openshift-monitoring/node-exporter-qfl7v"
Oct 30 20:38:15.847585 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:15.847546    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:15.849470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:15.849445    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ck9ws"
Oct 30 20:38:18.222233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:18.222206    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:18.793357 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:18.793315    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:18.891054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:18.891024    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:21.367837 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:21.367798    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:21.410486 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:21.410456    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:21.410718 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:21.410690    4852 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 20:38:21.556529 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:21.556498    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:38:38.909851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:38.909819    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-825zz"
Oct 30 20:38:39.088374 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:39.088342    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:39.164234 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:39.164137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:41.370507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:41.370471    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lhg4p"
Oct 30 20:38:43.113919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:43.113830    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bsg94"
Oct 30 20:38:43.801202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:43.801166    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:43.883426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:43.883382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:45.663259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:45.663223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:45.746232 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:45.746198    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:45.996375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:45.996288    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:46.087204 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:46.087171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:46.192481 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:46.192449    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:46.291280 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:46.291252    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:51.555700 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:38:51.555664    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:38:53.602275 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.602241    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 20:38:53.604690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.604671    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.608848 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.608820    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 20:38:53.609601 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609579    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 20:38:53.609697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609631    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 20:38:53.609697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609640    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 20:38:53.609697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609640    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wldr4\""
Oct 30 20:38:53.609697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609652    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3cp611nc7ognl\""
Oct 30 20:38:53.609697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609652    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 20:38:53.609697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609648    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 20:38:53.609697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609686    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 20:38:53.610022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609714    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 20:38:53.610022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609840    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 20:38:53.610022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.609880    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 20:38:53.611893 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.611874    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 20:38:53.612067 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.612048    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 20:38:53.612067 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.612063    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 20:38:53.616877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.613955    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 20:38:53.616877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.614256    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 20:38:53.623346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.623324    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 20:38:53.734208 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734168    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h7t4\" (UniqueName: \"kubernetes.io/projected/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-kube-api-access-8h7t4\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734208 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734211    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734239    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734335    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734363    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734408    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734463    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-config-out\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734480    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734537    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-config\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734579    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734648    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734671    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734691    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734797 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734715    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734797 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734733    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734797 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734749    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734797 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734774    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734911 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734827    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734911 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734866    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.734911 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.734882    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-web-config\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835480    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835533    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835551    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835571    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835597    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-config-out\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835622    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835649    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-config\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835671    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835706    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835730    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835745    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.835767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835760    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835779    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835800    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835826    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835856    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835885    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835913    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-web-config\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.835972    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h7t4\" (UniqueName: \"kubernetes.io/projected/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-kube-api-access-8h7t4\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.836001    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.836002    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.836051    4852 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 20:38:53.836753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.836338    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836815 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.836770    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.836815 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.836802    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.837146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.837121    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.839200 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.839172    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.839731 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.839708    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.839843 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.839820    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.840375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.840034    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-config-out\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.840375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.840142    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.840375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.840315    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.840375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.840315    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.840672 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.840573    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.840672 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.840596    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-config\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.840672 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.840649    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.841701 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.841677    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-web-config\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.841883 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.841864    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.841997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.841980    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.842453 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.842437    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.846602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.846583    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h7t4\" (UniqueName: \"kubernetes.io/projected/6ec22351-7ca8-448c-b9e1-fb43f8ec6257-kube-api-access-8h7t4\") pod \"prometheus-k8s-0\" (UID: \"6ec22351-7ca8-448c-b9e1-fb43f8ec6257\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:53.922640 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:53.922546    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:38:54.204466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:54.204286    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 20:38:54.208765 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:38:54.208735    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ec22351_7ca8_448c_b9e1_fb43f8ec6257.slice/crio-822d5032cdca3c03b7d96eda2fb4b43bd2805d5308b328578b3c94048f36d676 WatchSource:0}: Error finding container 822d5032cdca3c03b7d96eda2fb4b43bd2805d5308b328578b3c94048f36d676: Status 404 returned error can't find the container with id 822d5032cdca3c03b7d96eda2fb4b43bd2805d5308b328578b3c94048f36d676
Oct 30 20:38:54.210820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:54.210801    4852 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:38:55.186672 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:55.186632    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerStarted","Data":"822d5032cdca3c03b7d96eda2fb4b43bd2805d5308b328578b3c94048f36d676"}
Oct 30 20:38:56.191700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:56.191663    4852 generic.go:356] "Generic (PLEG): container finished" podID="6ec22351-7ca8-448c-b9e1-fb43f8ec6257" containerID="ca0e27a592c2dd46583cfb098d6998f5fc4fc1836b567cac46951f358d9bb2bc" exitCode=0
Oct 30 20:38:56.192171 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:56.191705    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerDied","Data":"ca0e27a592c2dd46583cfb098d6998f5fc4fc1836b567cac46951f358d9bb2bc"}
Oct 30 20:38:56.419952 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:56.419918    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:56.517325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:56.517287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:38:59.204764 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:59.204726    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerStarted","Data":"4e7dc8f6bbc30dbb387b06dd081d9a7fd1d6227433d9a2710d1cb414d0966ff3"}
Oct 30 20:38:59.205882 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:38:59.205852    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerStarted","Data":"3519236948c9754e0fa924e81dff2f47c7d59aeb203b9d7ce15e0e94174ee0e5"}
Oct 30 20:39:01.218246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:01.218166    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerStarted","Data":"01e5226281877204e018730d7b471842af026caf18997ee7009818b340e33fb5"}
Oct 30 20:39:01.218619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:01.218258    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerStarted","Data":"79e4156b966175aabb2f92f9c1c0ee9233c3cc66c863e9afe8c0af85680ab37a"}
Oct 30 20:39:01.218619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:01.218293    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerStarted","Data":"ce6f7b3f258eacad9f8e156057f53476b98d4068a21c54e59205fb4f714230d9"}
Oct 30 20:39:01.218619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:01.218320    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6ec22351-7ca8-448c-b9e1-fb43f8ec6257","Type":"ContainerStarted","Data":"e98345dea63e8a95cb5bbc1f1273cd95fcc14959a3b87d6db913a630517af3d2"}
Oct 30 20:39:01.247299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:01.247240    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.616455845 podStartE2EDuration="8.247226598s" totalImagesPullingTime="5.614495826s" totalInitContainerRuntime="16.274927ms" isStatefulPod=false podCreationTimestamp="2026-10-30 20:38:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:39:01.24583949 +0000 UTC m=+49.360573679" watchObservedRunningTime="2026-10-30 20:39:01.247226598 +0000 UTC m=+49.361960788"
Oct 30 20:39:02.663727 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:02.663694    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:39:03.745639 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:03.745604    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:39:03.847996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:03.847960    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:39:03.923192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:03.923157    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:39:04.020210 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:04.020179    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:39:21.555852 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:39:21.555818    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:39:34.315637 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:34.315586    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:39:34.334494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:34.334466    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:39:35.336461 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:35.336432    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 20:39:38.491258 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:39:38.491221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:39:51.556010 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:39:51.555971    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:40:02.716783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:40:02.716747    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:40:06.661653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:40:06.661620    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:40:09.918244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:40:09.918210    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:40:21.556632 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:40:21.556549    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:40:42.725725 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:40:42.725690    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:40:42.798121 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:40:42.798093    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:40:51.556359 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:40:51.556319    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:40:51.745424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:40:51.745373    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:40:59.919844 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:40:59.919811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:00.037091 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:00.037053    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:00.160999 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:00.160955    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:01.280587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:01.280552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:01.428054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:01.428014    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:01.990520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:01.990484    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"]
Oct 30 20:41:01.992592 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:01.992576    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.002688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.002649    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 20:41:02.002688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.002650    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 20:41:02.002688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.002689    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-gpw5r\""
Oct 30 20:41:02.003421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.003380    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"]
Oct 30 20:41:02.103537 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.103502    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"]
Oct 30 20:41:02.105930 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.105910    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.118030 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.118002    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"]
Oct 30 20:41:02.119584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.119566    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-xj6pt\""
Oct 30 20:41:02.145780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.145748    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmlmc\" (UniqueName: \"kubernetes.io/projected/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-kube-api-access-cmlmc\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.145925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.145844    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.145925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.145873    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.195472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.195442    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"]
Oct 30 20:41:02.197483 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.197466    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.203544 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.203351    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-6z7zr\""
Oct 30 20:41:02.207687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.207661    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"]
Oct 30 20:41:02.246524 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.246432    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.246524 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.246495    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmlmc\" (UniqueName: \"kubernetes.io/projected/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-kube-api-access-cmlmc\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.246743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.246586    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.246743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.246623    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.246743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.246648    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.246877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.246742    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsrns\" (UniqueName: \"kubernetes.io/projected/1d081644-ab43-4c44-b830-f0f925404b17-kube-api-access-xsrns\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.247069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.247041    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.247103 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.247067    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.257113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.257088    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmlmc\" (UniqueName: \"kubernetes.io/projected/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-kube-api-access-cmlmc\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.305006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.304975    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"]
Oct 30 20:41:02.307036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.307020    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.308893 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.308875    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:02.314020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.313916    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-fbmdr\""
Oct 30 20:41:02.318571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.318540    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"]
Oct 30 20:41:02.348304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.348128    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsrns\" (UniqueName: \"kubernetes.io/projected/1d081644-ab43-4c44-b830-f0f925404b17-kube-api-access-xsrns\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.348304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.348187    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.348304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.348225    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.348304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.348266    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74gnh\" (UniqueName: \"kubernetes.io/projected/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-kube-api-access-74gnh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.348606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.348368    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.348670 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.348647    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.348818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.348801    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.349042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.349024    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.361915 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.361890    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsrns\" (UniqueName: \"kubernetes.io/projected/1d081644-ab43-4c44-b830-f0f925404b17-kube-api-access-xsrns\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.401847 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.401813    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"]
Oct 30 20:41:02.404004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.403988    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.414775 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.414743    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-7pfgp\""
Oct 30 20:41:02.415258 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.415232    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"]
Oct 30 20:41:02.419821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.419796    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:02.449997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.449955    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74gnh\" (UniqueName: \"kubernetes.io/projected/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-kube-api-access-74gnh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.449997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.449996    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.450247 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.450029    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc466\" (UniqueName: \"kubernetes.io/projected/ec28443b-4625-41fd-8a5d-caf060a2d34b-kube-api-access-cc466\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.450247 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.450067    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.450247 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.450098    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.450247 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.450228    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.450518 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.450494    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.450576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.450539    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.459185 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.459158    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74gnh\" (UniqueName: \"kubernetes.io/projected/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-kube-api-access-74gnh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.509909 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.509882    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:02.548362 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.548317    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:02.551411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.551372    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.551499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.551480    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.551583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.551568    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc466\" (UniqueName: \"kubernetes.io/projected/ec28443b-4625-41fd-8a5d-caf060a2d34b-kube-api-access-cc466\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.551642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.551613    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.551696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.551670    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m87fm\" (UniqueName: \"kubernetes.io/projected/32025ff2-9c7a-4a3c-92e9-db3462c61075-kube-api-access-m87fm\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.551767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.551746    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.551887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.551871    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.552047 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.552028    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.561166 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.561138    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc466\" (UniqueName: \"kubernetes.io/projected/ec28443b-4625-41fd-8a5d-caf060a2d34b-kube-api-access-cc466\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.587989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.587956    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"]
Oct 30 20:41:02.591317 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:02.591291    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98b4f3b_e39c_4668_a8ca_1d06a83bed05.slice/crio-c23f13ac77ca24cb34d332a930177b4c142a5331a4db9410f96a883be368d7ba WatchSource:0}: Error finding container c23f13ac77ca24cb34d332a930177b4c142a5331a4db9410f96a883be368d7ba: Status 404 returned error can't find the container with id c23f13ac77ca24cb34d332a930177b4c142a5331a4db9410f96a883be368d7ba
Oct 30 20:41:02.637820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.637787    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:02.652821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.652796    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.652949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.652843    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m87fm\" (UniqueName: \"kubernetes.io/projected/32025ff2-9c7a-4a3c-92e9-db3462c61075-kube-api-access-m87fm\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.653021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.652958    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.653249 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.653225    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.653378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.653354    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.663098 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.663071    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m87fm\" (UniqueName: \"kubernetes.io/projected/32025ff2-9c7a-4a3c-92e9-db3462c61075-kube-api-access-m87fm\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.694179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.694147    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"]
Oct 30 20:41:02.697294 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:02.697271    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d081644_ab43_4c44_b830_f0f925404b17.slice/crio-28bc130ab56373f6d4c7fd81b7ae3a0142af411e970854387cc77cd68d1777c5 WatchSource:0}: Error finding container 28bc130ab56373f6d4c7fd81b7ae3a0142af411e970854387cc77cd68d1777c5: Status 404 returned error can't find the container with id 28bc130ab56373f6d4c7fd81b7ae3a0142af411e970854387cc77cd68d1777c5
Oct 30 20:41:02.720489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.720460    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:02.762246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.762154    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:02.789103 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.788984    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"]
Oct 30 20:41:02.793593 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:02.793559    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e9895a_f3e9_49c7_94d1_920a0c81dcf7.slice/crio-787a77d109a63690d5ed44625451473bd571cfa05ab81915053981d5b8b43564 WatchSource:0}: Error finding container 787a77d109a63690d5ed44625451473bd571cfa05ab81915053981d5b8b43564: Status 404 returned error can't find the container with id 787a77d109a63690d5ed44625451473bd571cfa05ab81915053981d5b8b43564
Oct 30 20:41:02.909309 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.909283    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"]
Oct 30 20:41:02.912344 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:02.912316    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec28443b_4625_41fd_8a5d_caf060a2d34b.slice/crio-87d73babc185689638236465b633f7820f2abed3e4a22dc2b25a7b11948b8c60 WatchSource:0}: Error finding container 87d73babc185689638236465b633f7820f2abed3e4a22dc2b25a7b11948b8c60: Status 404 returned error can't find the container with id 87d73babc185689638236465b633f7820f2abed3e4a22dc2b25a7b11948b8c60
Oct 30 20:41:02.989983 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:02.989946    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"]
Oct 30 20:41:02.993041 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:02.993012    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32025ff2_9c7a_4a3c_92e9_db3462c61075.slice/crio-aa16e80f82a735bc2df38d8a2b7829bdb9ee19b7377013a6fb39394547aa516d WatchSource:0}: Error finding container aa16e80f82a735bc2df38d8a2b7829bdb9ee19b7377013a6fb39394547aa516d: Status 404 returned error can't find the container with id aa16e80f82a735bc2df38d8a2b7829bdb9ee19b7377013a6fb39394547aa516d
Oct 30 20:41:03.598805 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:03.598624    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx" event={"ID":"32025ff2-9c7a-4a3c-92e9-db3462c61075","Type":"ContainerStarted","Data":"aa16e80f82a735bc2df38d8a2b7829bdb9ee19b7377013a6fb39394547aa516d"}
Oct 30 20:41:03.601412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:03.601362    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h" event={"ID":"ec28443b-4625-41fd-8a5d-caf060a2d34b","Type":"ContainerStarted","Data":"87d73babc185689638236465b633f7820f2abed3e4a22dc2b25a7b11948b8c60"}
Oct 30 20:41:03.604347 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:03.604317    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb" event={"ID":"1d081644-ab43-4c44-b830-f0f925404b17","Type":"ContainerStarted","Data":"28bc130ab56373f6d4c7fd81b7ae3a0142af411e970854387cc77cd68d1777c5"}
Oct 30 20:41:03.606464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:03.606378    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp" event={"ID":"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7","Type":"ContainerStarted","Data":"787a77d109a63690d5ed44625451473bd571cfa05ab81915053981d5b8b43564"}
Oct 30 20:41:03.607928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:03.607903    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m" event={"ID":"d98b4f3b-e39c-4668-a8ca-1d06a83bed05","Type":"ContainerStarted","Data":"c23f13ac77ca24cb34d332a930177b4c142a5331a4db9410f96a883be368d7ba"}
Oct 30 20:41:04.551693 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.551628    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"]
Oct 30 20:41:04.555319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.555292    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.561798 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.560945    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-b7xkn\""
Oct 30 20:41:04.563400 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.563306    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"]
Oct 30 20:41:04.600549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.600513    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.601001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.600576    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.601001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.600671    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nkzw\" (UniqueName: \"kubernetes.io/projected/b92104c8-4001-4003-80ae-7ad29ca336e0-kube-api-access-9nkzw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.703234 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.703191    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.703432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.703259    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nkzw\" (UniqueName: \"kubernetes.io/projected/b92104c8-4001-4003-80ae-7ad29ca336e0-kube-api-access-9nkzw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.703432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.703332    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.703818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.703794    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.703896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.703789    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.714272 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.714209    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nkzw\" (UniqueName: \"kubernetes.io/projected/b92104c8-4001-4003-80ae-7ad29ca336e0-kube-api-access-9nkzw\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:04.879000 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:04.878914    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:05.198780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:05.197723    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"]
Oct 30 20:41:05.214479 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:05.214405    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92104c8_4001_4003_80ae_7ad29ca336e0.slice/crio-72f5046e7c04219db5ac9e20c33b34f0a42fb9017633b65e8dacba9feb88426b WatchSource:0}: Error finding container 72f5046e7c04219db5ac9e20c33b34f0a42fb9017633b65e8dacba9feb88426b: Status 404 returned error can't find the container with id 72f5046e7c04219db5ac9e20c33b34f0a42fb9017633b65e8dacba9feb88426b
Oct 30 20:41:05.539969 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:05.539921    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:05.622078 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:05.622040    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9" event={"ID":"b92104c8-4001-4003-80ae-7ad29ca336e0","Type":"ContainerStarted","Data":"72f5046e7c04219db5ac9e20c33b34f0a42fb9017633b65e8dacba9feb88426b"}
Oct 30 20:41:05.639748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:05.639710    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:06.502835 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.502794    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"]
Oct 30 20:41:06.506423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.506321    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.517658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.517629    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 20:41:06.517792 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.517694    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-h8j6c\""
Oct 30 20:41:06.517792 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.517780    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 20:41:06.517909 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.517824    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520168    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520228    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520277    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520310    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520339    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520379    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520423    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520456    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.520759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.520477    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5ht7\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-kube-api-access-z5ht7\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.522825 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.522563    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"]
Oct 30 20:41:06.552163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.552066    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:06.607972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.607922    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"]
Oct 30 20:41:06.612489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.611771    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.617443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.617401    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-pptjr\""
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621766    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da498268-ef23-4b1d-812b-e88573f2268e-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621817    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621847    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da498268-ef23-4b1d-812b-e88573f2268e-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621869    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621890    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da498268-ef23-4b1d-812b-e88573f2268e-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621909    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621924    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5ht7\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-kube-api-access-z5ht7\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621948    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.621978    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622020    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622045    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622063    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622083    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt69l\" (UniqueName: \"kubernetes.io/projected/da498268-ef23-4b1d-812b-e88573f2268e-kube-api-access-wt69l\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622107    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622145    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622184    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622204    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.622881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.622223    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.633576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.624541    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.633576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.625153    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.633576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.625692    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.633576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.625754    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.633576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.626263    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.633576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.626489    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.633576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.628503    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"]
Oct 30 20:41:06.638868 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.638840    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5ht7\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-kube-api-access-z5ht7\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.647721 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.647691    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.653180 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.653154    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-2h8ll\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.668832 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.668756    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:06.723465 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723429    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723479    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723507    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt69l\" (UniqueName: \"kubernetes.io/projected/da498268-ef23-4b1d-812b-e88573f2268e-kube-api-access-wt69l\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723534    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723571    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723607    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da498268-ef23-4b1d-812b-e88573f2268e-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723635    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da498268-ef23-4b1d-812b-e88573f2268e-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723670    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da498268-ef23-4b1d-812b-e88573f2268e-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.723896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.723705    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.726099 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.725740    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da498268-ef23-4b1d-812b-e88573f2268e-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.728423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.726640    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.728423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.727930    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.728423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.728135    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.728423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.728355    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.729851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.729732    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da498268-ef23-4b1d-812b-e88573f2268e-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.731975 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.731932    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da498268-ef23-4b1d-812b-e88573f2268e-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.741079 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.741056    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt69l\" (UniqueName: \"kubernetes.io/projected/da498268-ef23-4b1d-812b-e88573f2268e-kube-api-access-wt69l\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.747560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.747501    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da498268-ef23-4b1d-812b-e88573f2268e-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-689986b98c-m284n\" (UID: \"da498268-ef23-4b1d-812b-e88573f2268e\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:06.771732 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.771698    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:06.829650 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.829619    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:06.922701 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.921010    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"]
Oct 30 20:41:06.927158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.926707    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:06.931381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.930627    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"]
Oct 30 20:41:06.933659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.932209    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-6gthp\""
Oct 30 20:41:06.942677 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:06.942656    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:07.027074 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.026814    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.027074 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.026878    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzcm7\" (UniqueName: \"kubernetes.io/projected/b1802967-ee76-4b7c-af13-6921d7d59822-kube-api-access-zzcm7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.027074 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.026945    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.128203 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.128150    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.128433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.128219    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.128433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.128280    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzcm7\" (UniqueName: \"kubernetes.io/projected/b1802967-ee76-4b7c-af13-6921d7d59822-kube-api-access-zzcm7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.129219 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.129192    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.129486 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.129444    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.141322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.141235    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"]
Oct 30 20:41:07.145193 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.145100    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzcm7\" (UniqueName: \"kubernetes.io/projected/b1802967-ee76-4b7c-af13-6921d7d59822-kube-api-access-zzcm7\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.148595 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:07.148549    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c9e9e4_3f97_4165_b06e_b15c24cd9eba.slice/crio-dd9e851293815f583b490c16daf7d0eac7a427484c09568fcd417c527dc647d4 WatchSource:0}: Error finding container dd9e851293815f583b490c16daf7d0eac7a427484c09568fcd417c527dc647d4: Status 404 returned error can't find the container with id dd9e851293815f583b490c16daf7d0eac7a427484c09568fcd417c527dc647d4
Oct 30 20:41:07.250774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.250740    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:07.294662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.294633    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"]
Oct 30 20:41:07.306024 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:07.305966    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda498268_ef23_4b1d_812b_e88573f2268e.slice/crio-7fcc32181f88131e73ca2aa42e4a69c2d73a880379e461ad03c9a88b7d1e7059 WatchSource:0}: Error finding container 7fcc32181f88131e73ca2aa42e4a69c2d73a880379e461ad03c9a88b7d1e7059: Status 404 returned error can't find the container with id 7fcc32181f88131e73ca2aa42e4a69c2d73a880379e461ad03c9a88b7d1e7059
Oct 30 20:41:07.583576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.583490    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"]
Oct 30 20:41:07.645618 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.645367    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8" event={"ID":"b1802967-ee76-4b7c-af13-6921d7d59822","Type":"ContainerStarted","Data":"d1e42e72593f5e12d01063747782e5d3c177958413985f4411ff0012f191d430"}
Oct 30 20:41:07.658755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.658624    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n" event={"ID":"da498268-ef23-4b1d-812b-e88573f2268e","Type":"ContainerStarted","Data":"7fcc32181f88131e73ca2aa42e4a69c2d73a880379e461ad03c9a88b7d1e7059"}
Oct 30 20:41:07.668489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:07.668290    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" event={"ID":"64c9e9e4-3f97-4165-b06e-b15c24cd9eba","Type":"ContainerStarted","Data":"dd9e851293815f583b490c16daf7d0eac7a427484c09568fcd417c527dc647d4"}
Oct 30 20:41:09.230819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.230780    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"]
Oct 30 20:41:09.237346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.236714    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.256422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.253192    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.256422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.253439    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mssq8\" (UniqueName: \"kubernetes.io/projected/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-kube-api-access-mssq8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.256422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.253694    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.271045 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.271003    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"]
Oct 30 20:41:09.274223 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.274193    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"]
Oct 30 20:41:09.281617 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.281593    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.289209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.288679    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"]
Oct 30 20:41:09.297925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.297896    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"]
Oct 30 20:41:09.303049 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.302602    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.312147 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.312048    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"]
Oct 30 20:41:09.342477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.340666    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"]
Oct 30 20:41:09.345157 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.345132    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.352916 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.352830    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"]
Oct 30 20:41:09.354911 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.354848    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.355157 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.355124    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.355726 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.355697    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmrjj\" (UniqueName: \"kubernetes.io/projected/35042d17-85c3-401a-87e3-8fa7164f9c8d-kube-api-access-cmrjj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.355881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.355863    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.356051 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.356035    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.356293 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.356273    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mssq8\" (UniqueName: \"kubernetes.io/projected/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-kube-api-access-mssq8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.356496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.356467    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.356673 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.356643    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m72h\" (UniqueName: \"kubernetes.io/projected/70404f30-0e3d-40f3-9865-bb17b198d7e6-kube-api-access-5m72h\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.359339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.358097    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.359339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.358517    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.359339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.358051    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.404619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.403783    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mssq8\" (UniqueName: \"kubernetes.io/projected/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-kube-api-access-mssq8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.459527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459337    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28zb5\" (UniqueName: \"kubernetes.io/projected/dbd673b6-9681-421c-911c-0b2f9db7256b-kube-api-access-28zb5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.459527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459433    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.459527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459483    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.459527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459518    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m72h\" (UniqueName: \"kubernetes.io/projected/70404f30-0e3d-40f3-9865-bb17b198d7e6-kube-api-access-5m72h\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.459786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459557    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.459786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459582    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.459786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459619    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmrjj\" (UniqueName: \"kubernetes.io/projected/35042d17-85c3-401a-87e3-8fa7164f9c8d-kube-api-access-cmrjj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.459786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459652    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.459786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.459680    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.460168 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.460111    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.460649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.460486    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.461130 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.461110    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.461382 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.461359    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.482992 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.482911    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m72h\" (UniqueName: \"kubernetes.io/projected/70404f30-0e3d-40f3-9865-bb17b198d7e6-kube-api-access-5m72h\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.484194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.484139    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmrjj\" (UniqueName: \"kubernetes.io/projected/35042d17-85c3-401a-87e3-8fa7164f9c8d-kube-api-access-cmrjj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.560699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.560659    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28zb5\" (UniqueName: \"kubernetes.io/projected/dbd673b6-9681-421c-911c-0b2f9db7256b-kube-api-access-28zb5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.560886 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.560748    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.560886 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.560804    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.561230 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.561205    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.561452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.561426    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.583329 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.583288    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28zb5\" (UniqueName: \"kubernetes.io/projected/dbd673b6-9681-421c-911c-0b2f9db7256b-kube-api-access-28zb5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.611047 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.610778    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:09.625667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.625457    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:09.636018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.635811    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:09.684530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.684449    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:09.710409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.710359    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:09.995256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:09.995169    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"]
Oct 30 20:41:10.000677 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:10.000643    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74738e1d_b3a7_4d9a_97a9_d8d1f437ea3b.slice/crio-5e1ec9fa683f7b71ea2a4179f54dba69ffa6636c71729c7f45d5167a4d344bb1 WatchSource:0}: Error finding container 5e1ec9fa683f7b71ea2a4179f54dba69ffa6636c71729c7f45d5167a4d344bb1: Status 404 returned error can't find the container with id 5e1ec9fa683f7b71ea2a4179f54dba69ffa6636c71729c7f45d5167a4d344bb1
Oct 30 20:41:10.051591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:10.051489    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"]
Oct 30 20:41:10.053969 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:10.052575    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"]
Oct 30 20:41:10.103367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:10.103334    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"]
Oct 30 20:41:10.112260 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:10.112226    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd673b6_9681_421c_911c_0b2f9db7256b.slice/crio-ca7b2cc98ef313bb016e8084bf5927d6db32e2e1ff0eafca575aae3080ea8dd6 WatchSource:0}: Error finding container ca7b2cc98ef313bb016e8084bf5927d6db32e2e1ff0eafca575aae3080ea8dd6: Status 404 returned error can't find the container with id ca7b2cc98ef313bb016e8084bf5927d6db32e2e1ff0eafca575aae3080ea8dd6
Oct 30 20:41:10.714673 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:10.714633    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb" event={"ID":"dbd673b6-9681-421c-911c-0b2f9db7256b","Type":"ContainerStarted","Data":"ca7b2cc98ef313bb016e8084bf5927d6db32e2e1ff0eafca575aae3080ea8dd6"}
Oct 30 20:41:10.735814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:10.735704    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl" event={"ID":"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b","Type":"ContainerStarted","Data":"5e1ec9fa683f7b71ea2a4179f54dba69ffa6636c71729c7f45d5167a4d344bb1"}
Oct 30 20:41:10.781927 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:10.781884    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk" event={"ID":"35042d17-85c3-401a-87e3-8fa7164f9c8d","Type":"ContainerStarted","Data":"88afc2a2d03d31a9f8f04207ad6a038a58420a267584b49effa22730901e1d75"}
Oct 30 20:41:10.799273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:10.799222    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8" event={"ID":"70404f30-0e3d-40f3-9865-bb17b198d7e6","Type":"ContainerStarted","Data":"749d654db9d537613aff1b7034a6f8c20253c909181b762f257d78887851955c"}
Oct 30 20:41:18.741652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:18.740780    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:41:18.741652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:18.740866    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:41:18.741652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:18.740909    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:41:19.323811 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.323761    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:41:19.323982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.323861    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:41:19.323982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.323910    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:41:19.893125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.893082    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n" event={"ID":"da498268-ef23-4b1d-812b-e88573f2268e","Type":"ContainerStarted","Data":"b6a61304cf48b0b769616e59ffd72921e287133052b50f66806bc4b8e518be29"}
Oct 30 20:41:19.905659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.905625    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" event={"ID":"64c9e9e4-3f97-4165-b06e-b15c24cd9eba","Type":"ContainerStarted","Data":"f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4"}
Oct 30 20:41:19.936303 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.936236    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n" podStartSLOduration=1.9232822939999998 podStartE2EDuration="13.936220635s" totalImagesPullingTime="12.012938341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:41:19.917062221 +0000 UTC m=+188.031796412" watchObservedRunningTime="2026-10-30 20:41:19.936220635 +0000 UTC m=+188.050954824"
Oct 30 20:41:19.937033 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.936989    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podStartSLOduration=2.353158439 podStartE2EDuration="13.936976262s" totalImagesPullingTime="11.583817823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:41:19.936013414 +0000 UTC m=+188.050747606" watchObservedRunningTime="2026-10-30 20:41:19.936976262 +0000 UTC m=+188.051710454"
Oct 30 20:41:19.943000 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:19.942947    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:20.832043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:20.830195    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:20.836519 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:20.836492    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:20.915462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:20.915158    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:20.916834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:20.916475    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:41:20.949498 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:20.949431    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:21.602011 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:41:21.601953    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:41:21.922660 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:21.922588    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:21.923617 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:21.923595    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-689986b98c-m284n"
Oct 30 20:41:23.938397 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.937441    4852 generic.go:356] "Generic (PLEG): container finished" podID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerID="b5b0937a0198129fc5d71a3eb0be87bd753c4adab2074197f56862ed3d2556dc" exitCode=0
Oct 30 20:41:23.938397 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.937542    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp" event={"ID":"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7","Type":"ContainerDied","Data":"b5b0937a0198129fc5d71a3eb0be87bd753c4adab2074197f56862ed3d2556dc"}
Oct 30 20:41:23.942043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.941602    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h" event={"ID":"ec28443b-4625-41fd-8a5d-caf060a2d34b","Type":"ContainerStarted","Data":"1d8f5b58bf09c77280c4f2afe0394771f4e3f26fdfd18bc5f3635b53ab0c8801"}
Oct 30 20:41:23.943665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.943637    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb" event={"ID":"dbd673b6-9681-421c-911c-0b2f9db7256b","Type":"ContainerStarted","Data":"25e39d5b432e810cf0ef6b5d531a8730b17a39cf4e175836f94d216d7aacc28a"}
Oct 30 20:41:23.945300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.945276    4852 generic.go:356] "Generic (PLEG): container finished" podID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerID="87655f44be821539096193181cab02b2f353467e1fe0b54090b02e65345c5c91" exitCode=0
Oct 30 20:41:23.945423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.945373    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m" event={"ID":"d98b4f3b-e39c-4668-a8ca-1d06a83bed05","Type":"ContainerDied","Data":"87655f44be821539096193181cab02b2f353467e1fe0b54090b02e65345c5c91"}
Oct 30 20:41:23.946982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.946959    4852 generic.go:356] "Generic (PLEG): container finished" podID="1d081644-ab43-4c44-b830-f0f925404b17" containerID="b902f62b256f47e9d7ba9dc1bcb53d9397cb42abb5d3e29677815ae4ea503fb0" exitCode=0
Oct 30 20:41:23.947084 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:23.947044    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb" event={"ID":"1d081644-ab43-4c44-b830-f0f925404b17","Type":"ContainerDied","Data":"b902f62b256f47e9d7ba9dc1bcb53d9397cb42abb5d3e29677815ae4ea503fb0"}
Oct 30 20:41:24.954077 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.954039    4852 generic.go:356] "Generic (PLEG): container finished" podID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerID="1d8f5b58bf09c77280c4f2afe0394771f4e3f26fdfd18bc5f3635b53ab0c8801" exitCode=0
Oct 30 20:41:24.954581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.954140    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h" event={"ID":"ec28443b-4625-41fd-8a5d-caf060a2d34b","Type":"ContainerDied","Data":"1d8f5b58bf09c77280c4f2afe0394771f4e3f26fdfd18bc5f3635b53ab0c8801"}
Oct 30 20:41:24.959514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.957010    4852 generic.go:356] "Generic (PLEG): container finished" podID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerID="71764e608be13a98850e1920417a2e7b27f24405a385136647f7162707e942d1" exitCode=0
Oct 30 20:41:24.959514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.957082    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9" event={"ID":"b92104c8-4001-4003-80ae-7ad29ca336e0","Type":"ContainerDied","Data":"71764e608be13a98850e1920417a2e7b27f24405a385136647f7162707e942d1"}
Oct 30 20:41:24.961017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.960999    4852 generic.go:356] "Generic (PLEG): container finished" podID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerID="25e39d5b432e810cf0ef6b5d531a8730b17a39cf4e175836f94d216d7aacc28a" exitCode=0
Oct 30 20:41:24.961105 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.961083    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb" event={"ID":"dbd673b6-9681-421c-911c-0b2f9db7256b","Type":"ContainerDied","Data":"25e39d5b432e810cf0ef6b5d531a8730b17a39cf4e175836f94d216d7aacc28a"}
Oct 30 20:41:24.963043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.962970    4852 generic.go:356] "Generic (PLEG): container finished" podID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerID="40f639279ab7cc4961d7f30f3bfe9cc36e0a235ddaf930d8302277c6133ae06a" exitCode=0
Oct 30 20:41:24.963117 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.963055    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl" event={"ID":"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b","Type":"ContainerDied","Data":"40f639279ab7cc4961d7f30f3bfe9cc36e0a235ddaf930d8302277c6133ae06a"}
Oct 30 20:41:24.965375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.965348    4852 generic.go:356] "Generic (PLEG): container finished" podID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerID="19848e5eb469ed7bb961a856f68034a20f0f8dc2089d790a3d4d4c8c92bc6a84" exitCode=0
Oct 30 20:41:24.965496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.965453    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx" event={"ID":"32025ff2-9c7a-4a3c-92e9-db3462c61075","Type":"ContainerDied","Data":"19848e5eb469ed7bb961a856f68034a20f0f8dc2089d790a3d4d4c8c92bc6a84"}
Oct 30 20:41:24.968413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.968338    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk" event={"ID":"35042d17-85c3-401a-87e3-8fa7164f9c8d","Type":"ContainerDied","Data":"1dd892c9f0c851fd118b9f344124b7238ce41a7ecfa76d2d631bb73211a7dfd5"}
Oct 30 20:41:24.968779 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.968758    4852 generic.go:356] "Generic (PLEG): container finished" podID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerID="1dd892c9f0c851fd118b9f344124b7238ce41a7ecfa76d2d631bb73211a7dfd5" exitCode=0
Oct 30 20:41:24.970654 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.970625    4852 generic.go:356] "Generic (PLEG): container finished" podID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerID="36e9cc1e35681d7f63a111ace6a52da96d83b0105b0532e98c49157a2bc3d242" exitCode=0
Oct 30 20:41:24.970760 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:24.970694    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8" event={"ID":"70404f30-0e3d-40f3-9865-bb17b198d7e6","Type":"ContainerDied","Data":"36e9cc1e35681d7f63a111ace6a52da96d83b0105b0532e98c49157a2bc3d242"}
Oct 30 20:41:25.977447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.977358    4852 generic.go:356] "Generic (PLEG): container finished" podID="1d081644-ab43-4c44-b830-f0f925404b17" containerID="ab409ecc44052ccba47ad27d065a49271a5bda477bb5aaad5e0ee3b99095857d" exitCode=0
Oct 30 20:41:25.977849 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.977449    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb" event={"ID":"1d081644-ab43-4c44-b830-f0f925404b17","Type":"ContainerDied","Data":"ab409ecc44052ccba47ad27d065a49271a5bda477bb5aaad5e0ee3b99095857d"}
Oct 30 20:41:25.980507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.980483    4852 generic.go:356] "Generic (PLEG): container finished" podID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerID="3e29b86d3c34a71ed32ace713e38575ab3d48000d5e4432b2db4e13e0b60d989" exitCode=0
Oct 30 20:41:25.980667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.980645    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp" event={"ID":"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7","Type":"ContainerDied","Data":"3e29b86d3c34a71ed32ace713e38575ab3d48000d5e4432b2db4e13e0b60d989"}
Oct 30 20:41:25.983198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.983178    4852 generic.go:356] "Generic (PLEG): container finished" podID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerID="50d1061ca695207d24c9b6502178c138fdd65aa201591e03c74adbf012d15d5a" exitCode=0
Oct 30 20:41:25.983295 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.983264    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl" event={"ID":"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b","Type":"ContainerDied","Data":"50d1061ca695207d24c9b6502178c138fdd65aa201591e03c74adbf012d15d5a"}
Oct 30 20:41:25.985060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.985035    4852 generic.go:356] "Generic (PLEG): container finished" podID="b1802967-ee76-4b7c-af13-6921d7d59822" containerID="e64e5d8c961355e15e8b7c28f51eb87a7950f0f3faec965eaae0b16bb1e74951" exitCode=0
Oct 30 20:41:25.985166 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.985132    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8" event={"ID":"b1802967-ee76-4b7c-af13-6921d7d59822","Type":"ContainerDied","Data":"e64e5d8c961355e15e8b7c28f51eb87a7950f0f3faec965eaae0b16bb1e74951"}
Oct 30 20:41:25.987079 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.987057    4852 generic.go:356] "Generic (PLEG): container finished" podID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerID="43226be01b0f5cb876900b4128357bfbc370d5dc7c2c25d982d41643afd81122" exitCode=0
Oct 30 20:41:25.987198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.987137    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk" event={"ID":"35042d17-85c3-401a-87e3-8fa7164f9c8d","Type":"ContainerDied","Data":"43226be01b0f5cb876900b4128357bfbc370d5dc7c2c25d982d41643afd81122"}
Oct 30 20:41:25.989333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.989309    4852 generic.go:356] "Generic (PLEG): container finished" podID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerID="7581282576c21c3286efdbbc8f308ebef7a34d90fab4a1af90cb51ab3f9b0217" exitCode=0
Oct 30 20:41:25.989475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.989422    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8" event={"ID":"70404f30-0e3d-40f3-9865-bb17b198d7e6","Type":"ContainerDied","Data":"7581282576c21c3286efdbbc8f308ebef7a34d90fab4a1af90cb51ab3f9b0217"}
Oct 30 20:41:25.991491 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.991468    4852 generic.go:356] "Generic (PLEG): container finished" podID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerID="551bdd1899df95d8eb8e01925de888365dc9188c1d948cc5df9b5d1fbaea8b31" exitCode=0
Oct 30 20:41:25.991594 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:25.991498    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m" event={"ID":"d98b4f3b-e39c-4668-a8ca-1d06a83bed05","Type":"ContainerDied","Data":"551bdd1899df95d8eb8e01925de888365dc9188c1d948cc5df9b5d1fbaea8b31"}
Oct 30 20:41:27.005837 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:27.005799    4852 generic.go:356] "Generic (PLEG): container finished" podID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerID="287463359db3033696c5a28060fde6c7fb723f7b301f940f2110134561c0aead" exitCode=0
Oct 30 20:41:27.006179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:27.005905    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h" event={"ID":"ec28443b-4625-41fd-8a5d-caf060a2d34b","Type":"ContainerDied","Data":"287463359db3033696c5a28060fde6c7fb723f7b301f940f2110134561c0aead"}
Oct 30 20:41:27.020182 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:27.020017    4852 generic.go:356] "Generic (PLEG): container finished" podID="b1802967-ee76-4b7c-af13-6921d7d59822" containerID="ea6e5eb97618923a2ce08fbef98d703a1a2ee5e050c06d628517765a7924cffb" exitCode=0
Oct 30 20:41:27.020182 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:27.020128    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8" event={"ID":"b1802967-ee76-4b7c-af13-6921d7d59822","Type":"ContainerDied","Data":"ea6e5eb97618923a2ce08fbef98d703a1a2ee5e050c06d628517765a7924cffb"}
Oct 30 20:41:28.049228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:28.049184    4852 generic.go:356] "Generic (PLEG): container finished" podID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerID="a889cae7367cdf97feb27020828c3bfbcec747ca9643eabe6691f74a52fd32af" exitCode=0
Oct 30 20:41:28.049721 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:28.049266    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb" event={"ID":"dbd673b6-9681-421c-911c-0b2f9db7256b","Type":"ContainerDied","Data":"a889cae7367cdf97feb27020828c3bfbcec747ca9643eabe6691f74a52fd32af"}
Oct 30 20:41:30.109690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:30.109438    4852 generic.go:356] "Generic (PLEG): container finished" podID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerID="146c20c203d4df87f50b2cccab3b6fd25e25600760b2596ae2290ca44dd66d22" exitCode=0
Oct 30 20:41:30.109690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:30.109623    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9" event={"ID":"b92104c8-4001-4003-80ae-7ad29ca336e0","Type":"ContainerDied","Data":"146c20c203d4df87f50b2cccab3b6fd25e25600760b2596ae2290ca44dd66d22"}
Oct 30 20:41:32.135542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:32.135463    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx" event={"ID":"32025ff2-9c7a-4a3c-92e9-db3462c61075","Type":"ContainerStarted","Data":"afb1230db76058170578cdbcac26f478eebbcccf19b1f68dd11ad58d6df73dad"}
Oct 30 20:41:33.159884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:33.159842    4852 generic.go:356] "Generic (PLEG): container finished" podID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerID="afb1230db76058170578cdbcac26f478eebbcccf19b1f68dd11ad58d6df73dad" exitCode=0
Oct 30 20:41:33.160356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:33.159962    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx" event={"ID":"32025ff2-9c7a-4a3c-92e9-db3462c61075","Type":"ContainerDied","Data":"afb1230db76058170578cdbcac26f478eebbcccf19b1f68dd11ad58d6df73dad"}
Oct 30 20:41:48.330201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:48.329586    4852 generic.go:356] "Generic (PLEG): container finished" podID="1d081644-ab43-4c44-b830-f0f925404b17" containerID="fc666508dec4ed4ae6fafc5e51c2185ee39e3270121ca8152061d692368cea66" exitCode=0
Oct 30 20:41:48.330201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:48.329712    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb" event={"ID":"1d081644-ab43-4c44-b830-f0f925404b17","Type":"ContainerDied","Data":"fc666508dec4ed4ae6fafc5e51c2185ee39e3270121ca8152061d692368cea66"}
Oct 30 20:41:49.341336 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.340873    4852 generic.go:356] "Generic (PLEG): container finished" podID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerID="4510e5cf07684fc035843baae9a632af110f907fe49dbeee47f48d7443c63487" exitCode=0
Oct 30 20:41:49.341336 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.341056    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp" event={"ID":"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7","Type":"ContainerDied","Data":"4510e5cf07684fc035843baae9a632af110f907fe49dbeee47f48d7443c63487"}
Oct 30 20:41:49.343686 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.343658    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk" event={"ID":"35042d17-85c3-401a-87e3-8fa7164f9c8d","Type":"ContainerStarted","Data":"aa3033cc28c00bb1fbf134feb114f1b9bce4eda0f3c3aa406b288274ca24bab8"}
Oct 30 20:41:49.351455 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.350887    4852 generic.go:356] "Generic (PLEG): container finished" podID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerID="80d4aaa5843cc34841f46102f095d5a2eedb382eaa58adc979f68c63cb1e9536" exitCode=0
Oct 30 20:41:49.351455 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.351379    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8" event={"ID":"70404f30-0e3d-40f3-9865-bb17b198d7e6","Type":"ContainerDied","Data":"80d4aaa5843cc34841f46102f095d5a2eedb382eaa58adc979f68c63cb1e9536"}
Oct 30 20:41:49.411654 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.410590    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk" podStartSLOduration=2.8284760540000002 podStartE2EDuration="40.410571262s" totalImagesPullingTime="37.582095208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:41:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:41:49.4087723 +0000 UTC m=+217.523506521" watchObservedRunningTime="2026-10-30 20:41:49.410571262 +0000 UTC m=+217.525305454"
Oct 30 20:41:49.541407 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.541073    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:49.626861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.626825    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-util\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-util\") pod \"1d081644-ab43-4c44-b830-f0f925404b17\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") "
Oct 30 20:41:49.627020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.626919    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-bundle\") pod \"1d081644-ab43-4c44-b830-f0f925404b17\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") "
Oct 30 20:41:49.627020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.626955    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d081644-ab43-4c44-b830-f0f925404b17-kube-api-access-xsrns\" (UniqueName: \"kubernetes.io/projected/1d081644-ab43-4c44-b830-f0f925404b17-kube-api-access-xsrns\") pod \"1d081644-ab43-4c44-b830-f0f925404b17\" (UID: \"1d081644-ab43-4c44-b830-f0f925404b17\") "
Oct 30 20:41:49.627633 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.627603    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-bundle" pod "1d081644-ab43-4c44-b830-f0f925404b17" (UID: "1d081644-ab43-4c44-b830-f0f925404b17"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:49.632877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.632845    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d081644-ab43-4c44-b830-f0f925404b17-kube-api-access-xsrns" pod "1d081644-ab43-4c44-b830-f0f925404b17" (UID: "1d081644-ab43-4c44-b830-f0f925404b17"). InnerVolumeSpecName "kube-api-access-xsrns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:49.633711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.633676    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-util" pod "1d081644-ab43-4c44-b830-f0f925404b17" (UID: "1d081644-ab43-4c44-b830-f0f925404b17"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:49.728608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.728576    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:49.728771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.728616    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsrns\" (UniqueName: \"kubernetes.io/projected/1d081644-ab43-4c44-b830-f0f925404b17-kube-api-access-xsrns\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:49.728771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:49.728634    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d081644-ab43-4c44-b830-f0f925404b17-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:50.358878 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.358287    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb"
Oct 30 20:41:50.358878 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.358278    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aqllgb" event={"ID":"1d081644-ab43-4c44-b830-f0f925404b17","Type":"ContainerDied","Data":"28bc130ab56373f6d4c7fd81b7ae3a0142af411e970854387cc77cd68d1777c5"}
Oct 30 20:41:50.358878 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.358431    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28bc130ab56373f6d4c7fd81b7ae3a0142af411e970854387cc77cd68d1777c5"
Oct 30 20:41:50.361743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.361590    4852 generic.go:356] "Generic (PLEG): container finished" podID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerID="d1e9217d8788a6316a4832cb8b3c9c27a201658ecb38f43a23a20b99280e4532" exitCode=0
Oct 30 20:41:50.361743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.361685    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h" event={"ID":"ec28443b-4625-41fd-8a5d-caf060a2d34b","Type":"ContainerDied","Data":"d1e9217d8788a6316a4832cb8b3c9c27a201658ecb38f43a23a20b99280e4532"}
Oct 30 20:41:50.364475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.364451    4852 generic.go:356] "Generic (PLEG): container finished" podID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerID="52f9b7dbce002ada38855fa684f5a0b72495d7b2210ae37ee99cce4146e4d378" exitCode=0
Oct 30 20:41:50.364622 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.364540    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl" event={"ID":"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b","Type":"ContainerDied","Data":"52f9b7dbce002ada38855fa684f5a0b72495d7b2210ae37ee99cce4146e4d378"}
Oct 30 20:41:50.367381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.367354    4852 generic.go:356] "Generic (PLEG): container finished" podID="b1802967-ee76-4b7c-af13-6921d7d59822" containerID="ff2143c41cfd2ecdc495f449345fa3229638b3997c8469a0a94b7c18244e9783" exitCode=0
Oct 30 20:41:50.367519 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.367486    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8" event={"ID":"b1802967-ee76-4b7c-af13-6921d7d59822","Type":"ContainerDied","Data":"ff2143c41cfd2ecdc495f449345fa3229638b3997c8469a0a94b7c18244e9783"}
Oct 30 20:41:50.370222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.370190    4852 generic.go:356] "Generic (PLEG): container finished" podID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerID="aa3033cc28c00bb1fbf134feb114f1b9bce4eda0f3c3aa406b288274ca24bab8" exitCode=0
Oct 30 20:41:50.370327 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.370230    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk" event={"ID":"35042d17-85c3-401a-87e3-8fa7164f9c8d","Type":"ContainerDied","Data":"aa3033cc28c00bb1fbf134feb114f1b9bce4eda0f3c3aa406b288274ca24bab8"}
Oct 30 20:41:50.373003 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.372976    4852 generic.go:356] "Generic (PLEG): container finished" podID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerID="bdab8a320da7e595b70bf12fe75eae9ea6f570f4254a53d593cf98fc3e33233c" exitCode=0
Oct 30 20:41:50.373100 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.373058    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m" event={"ID":"d98b4f3b-e39c-4668-a8ca-1d06a83bed05","Type":"ContainerDied","Data":"bdab8a320da7e595b70bf12fe75eae9ea6f570f4254a53d593cf98fc3e33233c"}
Oct 30 20:41:50.557300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.557270    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:50.585407 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.585135    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:50.638442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.638413    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-util\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-util\") pod \"70404f30-0e3d-40f3-9865-bb17b198d7e6\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") "
Oct 30 20:41:50.638581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.638482    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-kube-api-access-74gnh\" (UniqueName: \"kubernetes.io/projected/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-kube-api-access-74gnh\") pod \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") "
Oct 30 20:41:50.638581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.638559    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70404f30-0e3d-40f3-9865-bb17b198d7e6-kube-api-access-5m72h\" (UniqueName: \"kubernetes.io/projected/70404f30-0e3d-40f3-9865-bb17b198d7e6-kube-api-access-5m72h\") pod \"70404f30-0e3d-40f3-9865-bb17b198d7e6\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") "
Oct 30 20:41:50.638717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.638607    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-bundle\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-bundle\") pod \"70404f30-0e3d-40f3-9865-bb17b198d7e6\" (UID: \"70404f30-0e3d-40f3-9865-bb17b198d7e6\") "
Oct 30 20:41:50.638717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.638642    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-bundle\") pod \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") "
Oct 30 20:41:50.638717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.638686    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-util\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-util\") pod \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\" (UID: \"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7\") "
Oct 30 20:41:50.640147 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.640111    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-bundle" pod "70404f30-0e3d-40f3-9865-bb17b198d7e6" (UID: "70404f30-0e3d-40f3-9865-bb17b198d7e6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:50.640541 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.640499    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-bundle" pod "c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" (UID: "c8e9895a-f3e9-49c7-94d1-920a0c81dcf7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:50.641858 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.641824    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70404f30-0e3d-40f3-9865-bb17b198d7e6-kube-api-access-5m72h" pod "70404f30-0e3d-40f3-9865-bb17b198d7e6" (UID: "70404f30-0e3d-40f3-9865-bb17b198d7e6"). InnerVolumeSpecName "kube-api-access-5m72h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:50.642504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.642480    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-kube-api-access-74gnh" pod "c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" (UID: "c8e9895a-f3e9-49c7-94d1-920a0c81dcf7"). InnerVolumeSpecName "kube-api-access-74gnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:50.647509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.647478    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-util" pod "70404f30-0e3d-40f3-9865-bb17b198d7e6" (UID: "70404f30-0e3d-40f3-9865-bb17b198d7e6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:50.647884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.647853    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-util" pod "c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" (UID: "c8e9895a-f3e9-49c7-94d1-920a0c81dcf7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:50.739915 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.739860    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m72h\" (UniqueName: \"kubernetes.io/projected/70404f30-0e3d-40f3-9865-bb17b198d7e6-kube-api-access-5m72h\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:50.740086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.739929    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:50.740086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.739950    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:50.740086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.739963    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:50.740086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.739977    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70404f30-0e3d-40f3-9865-bb17b198d7e6-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:50.740086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:50.739991    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74gnh\" (UniqueName: \"kubernetes.io/projected/c8e9895a-f3e9-49c7-94d1-920a0c81dcf7-kube-api-access-74gnh\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.380361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.380325    4852 generic.go:356] "Generic (PLEG): container finished" podID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerID="b6e103ac2898310fb50e3a87538c3bd34b01992cc488c84cd66872e74bb5ec93" exitCode=0
Oct 30 20:41:51.380808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.380365    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9" event={"ID":"b92104c8-4001-4003-80ae-7ad29ca336e0","Type":"ContainerDied","Data":"b6e103ac2898310fb50e3a87538c3bd34b01992cc488c84cd66872e74bb5ec93"}
Oct 30 20:41:51.383370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.383342    4852 generic.go:356] "Generic (PLEG): container finished" podID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerID="50fa87ab003fabbd2729290a7628a87a7dc27090e2bb42bdc036a0ea6d3075f3" exitCode=0
Oct 30 20:41:51.383520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.383410    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb" event={"ID":"dbd673b6-9681-421c-911c-0b2f9db7256b","Type":"ContainerDied","Data":"50fa87ab003fabbd2729290a7628a87a7dc27090e2bb42bdc036a0ea6d3075f3"}
Oct 30 20:41:51.385744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.385726    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8"
Oct 30 20:41:51.385865 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.385755    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a78tm8" event={"ID":"70404f30-0e3d-40f3-9865-bb17b198d7e6","Type":"ContainerDied","Data":"749d654db9d537613aff1b7034a6f8c20253c909181b762f257d78887851955c"}
Oct 30 20:41:51.385865 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.385804    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="749d654db9d537613aff1b7034a6f8c20253c909181b762f257d78887851955c"
Oct 30 20:41:51.389278 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.389250    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp" event={"ID":"c8e9895a-f3e9-49c7-94d1-920a0c81dcf7","Type":"ContainerDied","Data":"787a77d109a63690d5ed44625451473bd571cfa05ab81915053981d5b8b43564"}
Oct 30 20:41:51.389361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.389295    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="787a77d109a63690d5ed44625451473bd571cfa05ab81915053981d5b8b43564"
Oct 30 20:41:51.390027 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.389944    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lj8qp"
Oct 30 20:41:51.557161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.557136    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:51.565601 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:41:51.565567    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:41:51.640639 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.640616    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:51.644262 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.644240    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:51.648158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.648136    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:51.650724 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.650701    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35042d17-85c3-401a-87e3-8fa7164f9c8d-kube-api-access-cmrjj\" (UniqueName: \"kubernetes.io/projected/35042d17-85c3-401a-87e3-8fa7164f9c8d-kube-api-access-cmrjj\") pod \"35042d17-85c3-401a-87e3-8fa7164f9c8d\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") "
Oct 30 20:41:51.650825 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.650782    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-util\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-util\") pod \"35042d17-85c3-401a-87e3-8fa7164f9c8d\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") "
Oct 30 20:41:51.650878 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.650866    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-bundle\") pod \"35042d17-85c3-401a-87e3-8fa7164f9c8d\" (UID: \"35042d17-85c3-401a-87e3-8fa7164f9c8d\") "
Oct 30 20:41:51.651590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.651563    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-bundle" pod "35042d17-85c3-401a-87e3-8fa7164f9c8d" (UID: "35042d17-85c3-401a-87e3-8fa7164f9c8d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.652654 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.652636    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:51.653001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.652976    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35042d17-85c3-401a-87e3-8fa7164f9c8d-kube-api-access-cmrjj" pod "35042d17-85c3-401a-87e3-8fa7164f9c8d" (UID: "35042d17-85c3-401a-87e3-8fa7164f9c8d"). InnerVolumeSpecName "kube-api-access-cmrjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:51.657024 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.657003    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-util" pod "35042d17-85c3-401a-87e3-8fa7164f9c8d" (UID: "35042d17-85c3-401a-87e3-8fa7164f9c8d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.748304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.748265    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:51.752102 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752076    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-bundle\") pod \"ec28443b-4625-41fd-8a5d-caf060a2d34b\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") "
Oct 30 20:41:51.752256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752123    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-util\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-util\") pod \"b1802967-ee76-4b7c-af13-6921d7d59822\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") "
Oct 30 20:41:51.752256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752181    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-util\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-util\") pod \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") "
Oct 30 20:41:51.752256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752224    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-bundle\") pod \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") "
Oct 30 20:41:51.752256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752255    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-bundle\") pod \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") "
Oct 30 20:41:51.752521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752277    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-kube-api-access-mssq8\" (UniqueName: \"kubernetes.io/projected/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-kube-api-access-mssq8\") pod \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\" (UID: \"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b\") "
Oct 30 20:41:51.752521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752303    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec28443b-4625-41fd-8a5d-caf060a2d34b-kube-api-access-cc466\" (UniqueName: \"kubernetes.io/projected/ec28443b-4625-41fd-8a5d-caf060a2d34b-kube-api-access-cc466\") pod \"ec28443b-4625-41fd-8a5d-caf060a2d34b\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") "
Oct 30 20:41:51.752521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752353    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1802967-ee76-4b7c-af13-6921d7d59822-kube-api-access-zzcm7\" (UniqueName: \"kubernetes.io/projected/b1802967-ee76-4b7c-af13-6921d7d59822-kube-api-access-zzcm7\") pod \"b1802967-ee76-4b7c-af13-6921d7d59822\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") "
Oct 30 20:41:51.752521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752381    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-util\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-util\") pod \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") "
Oct 30 20:41:51.752772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752747    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-bundle" pod "74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" (UID: "74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.753021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.752997    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-kube-api-access-cmlmc\" (UniqueName: \"kubernetes.io/projected/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-kube-api-access-cmlmc\") pod \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\" (UID: \"d98b4f3b-e39c-4668-a8ca-1d06a83bed05\") "
Oct 30 20:41:51.753021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753012    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-bundle" pod "d98b4f3b-e39c-4668-a8ca-1d06a83bed05" (UID: "d98b4f3b-e39c-4668-a8ca-1d06a83bed05"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.753194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753048    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-util\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-util\") pod \"ec28443b-4625-41fd-8a5d-caf060a2d34b\" (UID: \"ec28443b-4625-41fd-8a5d-caf060a2d34b\") "
Oct 30 20:41:51.753194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753094    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-bundle\") pod \"b1802967-ee76-4b7c-af13-6921d7d59822\" (UID: \"b1802967-ee76-4b7c-af13-6921d7d59822\") "
Oct 30 20:41:51.753370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753353    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmrjj\" (UniqueName: \"kubernetes.io/projected/35042d17-85c3-401a-87e3-8fa7164f9c8d-kube-api-access-cmrjj\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.753469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753373    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.753469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753400    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.753469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753413    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.753469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.753425    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35042d17-85c3-401a-87e3-8fa7164f9c8d-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.754929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.754903    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-bundle" pod "ec28443b-4625-41fd-8a5d-caf060a2d34b" (UID: "ec28443b-4625-41fd-8a5d-caf060a2d34b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.755427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.755374    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-kube-api-access-mssq8" pod "74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" (UID: "74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b"). InnerVolumeSpecName "kube-api-access-mssq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:51.755553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.755523    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-bundle" pod "b1802967-ee76-4b7c-af13-6921d7d59822" (UID: "b1802967-ee76-4b7c-af13-6921d7d59822"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.755615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.755558    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-kube-api-access-cmlmc" pod "d98b4f3b-e39c-4668-a8ca-1d06a83bed05" (UID: "d98b4f3b-e39c-4668-a8ca-1d06a83bed05"). InnerVolumeSpecName "kube-api-access-cmlmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:51.755674 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.755651    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1802967-ee76-4b7c-af13-6921d7d59822-kube-api-access-zzcm7" pod "b1802967-ee76-4b7c-af13-6921d7d59822" (UID: "b1802967-ee76-4b7c-af13-6921d7d59822"). InnerVolumeSpecName "kube-api-access-zzcm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:51.755755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.755739    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec28443b-4625-41fd-8a5d-caf060a2d34b-kube-api-access-cc466" pod "ec28443b-4625-41fd-8a5d-caf060a2d34b" (UID: "ec28443b-4625-41fd-8a5d-caf060a2d34b"). InnerVolumeSpecName "kube-api-access-cc466". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:51.758777 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.758742    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-util" pod "74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" (UID: "74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.761379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.761350    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-util" pod "d98b4f3b-e39c-4668-a8ca-1d06a83bed05" (UID: "d98b4f3b-e39c-4668-a8ca-1d06a83bed05"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.762629 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.762601    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-util" pod "b1802967-ee76-4b7c-af13-6921d7d59822" (UID: "b1802967-ee76-4b7c-af13-6921d7d59822"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.762992 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.762969    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-util" pod "ec28443b-4625-41fd-8a5d-caf060a2d34b" (UID: "ec28443b-4625-41fd-8a5d-caf060a2d34b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:51.854793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854722    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mssq8\" (UniqueName: \"kubernetes.io/projected/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-kube-api-access-mssq8\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.854793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854749    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc466\" (UniqueName: \"kubernetes.io/projected/ec28443b-4625-41fd-8a5d-caf060a2d34b-kube-api-access-cc466\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.854793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854759    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzcm7\" (UniqueName: \"kubernetes.io/projected/b1802967-ee76-4b7c-af13-6921d7d59822-kube-api-access-zzcm7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.854793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854768    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.854793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854778    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmlmc\" (UniqueName: \"kubernetes.io/projected/d98b4f3b-e39c-4668-a8ca-1d06a83bed05-kube-api-access-cmlmc\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.854793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854787    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.854793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854796    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.855111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854805    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec28443b-4625-41fd-8a5d-caf060a2d34b-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.855111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854812    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1802967-ee76-4b7c-af13-6921d7d59822-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:51.855111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:51.854822    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:52.395889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.395848    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk" event={"ID":"35042d17-85c3-401a-87e3-8fa7164f9c8d","Type":"ContainerDied","Data":"88afc2a2d03d31a9f8f04207ad6a038a58420a267584b49effa22730901e1d75"}
Oct 30 20:41:52.396354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.395931    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88afc2a2d03d31a9f8f04207ad6a038a58420a267584b49effa22730901e1d75"
Oct 30 20:41:52.396354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.395862    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crv8fk"
Oct 30 20:41:52.398013 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.397990    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m"
Oct 30 20:41:52.398123 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.398003    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckz49m" event={"ID":"d98b4f3b-e39c-4668-a8ca-1d06a83bed05","Type":"ContainerDied","Data":"c23f13ac77ca24cb34d332a930177b4c142a5331a4db9410f96a883be368d7ba"}
Oct 30 20:41:52.398280 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.398262    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c23f13ac77ca24cb34d332a930177b4c142a5331a4db9410f96a883be368d7ba"
Oct 30 20:41:52.400379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.400353    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h" event={"ID":"ec28443b-4625-41fd-8a5d-caf060a2d34b","Type":"ContainerDied","Data":"87d73babc185689638236465b633f7820f2abed3e4a22dc2b25a7b11948b8c60"}
Oct 30 20:41:52.400510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.400409    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87d73babc185689638236465b633f7820f2abed3e4a22dc2b25a7b11948b8c60"
Oct 30 20:41:52.400510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.400451    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ftr5h"
Oct 30 20:41:52.402328 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.402302    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl" event={"ID":"74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b","Type":"ContainerDied","Data":"5e1ec9fa683f7b71ea2a4179f54dba69ffa6636c71729c7f45d5167a4d344bb1"}
Oct 30 20:41:52.402698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.402669    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e1ec9fa683f7b71ea2a4179f54dba69ffa6636c71729c7f45d5167a4d344bb1"
Oct 30 20:41:52.402698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.402310    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74cbzl"
Oct 30 20:41:52.404457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.404431    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8" event={"ID":"b1802967-ee76-4b7c-af13-6921d7d59822","Type":"ContainerDied","Data":"d1e42e72593f5e12d01063747782e5d3c177958413985f4411ff0012f191d430"}
Oct 30 20:41:52.404536 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.404492    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1e42e72593f5e12d01063747782e5d3c177958413985f4411ff0012f191d430"
Oct 30 20:41:52.404536 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.404442    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be7154dp8"
Oct 30 20:41:52.406900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.406878    4852 generic.go:356] "Generic (PLEG): container finished" podID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerID="5dacc6a9f14b3824f264024859d24fa0550d44c42b488ef563fb3647cf53fea4" exitCode=0
Oct 30 20:41:52.406996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.406936    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx" event={"ID":"32025ff2-9c7a-4a3c-92e9-db3462c61075","Type":"ContainerDied","Data":"5dacc6a9f14b3824f264024859d24fa0550d44c42b488ef563fb3647cf53fea4"}
Oct 30 20:41:52.546325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.546300    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:52.548949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.548932    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:52.611006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.610972    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:52.661422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.661340    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-bundle\") pod \"dbd673b6-9681-421c-911c-0b2f9db7256b\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") "
Oct 30 20:41:52.661422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.661393    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b92104c8-4001-4003-80ae-7ad29ca336e0-kube-api-access-9nkzw\" (UniqueName: \"kubernetes.io/projected/b92104c8-4001-4003-80ae-7ad29ca336e0-kube-api-access-9nkzw\") pod \"b92104c8-4001-4003-80ae-7ad29ca336e0\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") "
Oct 30 20:41:52.661614 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.661433    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-bundle\") pod \"b92104c8-4001-4003-80ae-7ad29ca336e0\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") "
Oct 30 20:41:52.661614 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.661459    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbd673b6-9681-421c-911c-0b2f9db7256b-kube-api-access-28zb5\" (UniqueName: \"kubernetes.io/projected/dbd673b6-9681-421c-911c-0b2f9db7256b-kube-api-access-28zb5\") pod \"dbd673b6-9681-421c-911c-0b2f9db7256b\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") "
Oct 30 20:41:52.661614 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.661499    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-util\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-util\") pod \"dbd673b6-9681-421c-911c-0b2f9db7256b\" (UID: \"dbd673b6-9681-421c-911c-0b2f9db7256b\") "
Oct 30 20:41:52.661614 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.661527    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-util\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-util\") pod \"b92104c8-4001-4003-80ae-7ad29ca336e0\" (UID: \"b92104c8-4001-4003-80ae-7ad29ca336e0\") "
Oct 30 20:41:52.661951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.661919    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-bundle" pod "dbd673b6-9681-421c-911c-0b2f9db7256b" (UID: "dbd673b6-9681-421c-911c-0b2f9db7256b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:52.662212 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.662183    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-bundle" pod "b92104c8-4001-4003-80ae-7ad29ca336e0" (UID: "b92104c8-4001-4003-80ae-7ad29ca336e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:52.663631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.663608    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b92104c8-4001-4003-80ae-7ad29ca336e0-kube-api-access-9nkzw" pod "b92104c8-4001-4003-80ae-7ad29ca336e0" (UID: "b92104c8-4001-4003-80ae-7ad29ca336e0"). InnerVolumeSpecName "kube-api-access-9nkzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:52.663936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.663913    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbd673b6-9681-421c-911c-0b2f9db7256b-kube-api-access-28zb5" pod "dbd673b6-9681-421c-911c-0b2f9db7256b" (UID: "dbd673b6-9681-421c-911c-0b2f9db7256b"). InnerVolumeSpecName "kube-api-access-28zb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:52.666288 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.666266    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-util" pod "b92104c8-4001-4003-80ae-7ad29ca336e0" (UID: "b92104c8-4001-4003-80ae-7ad29ca336e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:52.667810 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.667790    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-util" pod "dbd673b6-9681-421c-911c-0b2f9db7256b" (UID: "dbd673b6-9681-421c-911c-0b2f9db7256b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:52.762421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.762372    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:52.762421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.762415    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nkzw\" (UniqueName: \"kubernetes.io/projected/b92104c8-4001-4003-80ae-7ad29ca336e0-kube-api-access-9nkzw\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:52.762421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.762426    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:52.762648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.762435    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28zb5\" (UniqueName: \"kubernetes.io/projected/dbd673b6-9681-421c-911c-0b2f9db7256b-kube-api-access-28zb5\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:52.762648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.762443    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbd673b6-9681-421c-911c-0b2f9db7256b-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:52.762648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:52.762452    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b92104c8-4001-4003-80ae-7ad29ca336e0-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:53.412924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.412893    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9" event={"ID":"b92104c8-4001-4003-80ae-7ad29ca336e0","Type":"ContainerDied","Data":"72f5046e7c04219db5ac9e20c33b34f0a42fb9017633b65e8dacba9feb88426b"}
Oct 30 20:41:53.413411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.412940    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72f5046e7c04219db5ac9e20c33b34f0a42fb9017633b65e8dacba9feb88426b"
Oct 30 20:41:53.413411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.412905    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cjwkt9"
Oct 30 20:41:53.414896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.414879    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb"
Oct 30 20:41:53.414982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.414921    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6gq5gb" event={"ID":"dbd673b6-9681-421c-911c-0b2f9db7256b","Type":"ContainerDied","Data":"ca7b2cc98ef313bb016e8084bf5927d6db32e2e1ff0eafca575aae3080ea8dd6"}
Oct 30 20:41:53.414982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.414967    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca7b2cc98ef313bb016e8084bf5927d6db32e2e1ff0eafca575aae3080ea8dd6"
Oct 30 20:41:53.530647 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.530624    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:53.568408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.568347    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-bundle\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-bundle\") pod \"32025ff2-9c7a-4a3c-92e9-db3462c61075\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") "
Oct 30 20:41:53.568587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.568466    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32025ff2-9c7a-4a3c-92e9-db3462c61075-kube-api-access-m87fm\" (UniqueName: \"kubernetes.io/projected/32025ff2-9c7a-4a3c-92e9-db3462c61075-kube-api-access-m87fm\") pod \"32025ff2-9c7a-4a3c-92e9-db3462c61075\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") "
Oct 30 20:41:53.568587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.568534    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-util\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-util\") pod \"32025ff2-9c7a-4a3c-92e9-db3462c61075\" (UID: \"32025ff2-9c7a-4a3c-92e9-db3462c61075\") "
Oct 30 20:41:53.568819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.568790    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-bundle" pod "32025ff2-9c7a-4a3c-92e9-db3462c61075" (UID: "32025ff2-9c7a-4a3c-92e9-db3462c61075"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:53.571135 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.570902    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32025ff2-9c7a-4a3c-92e9-db3462c61075-kube-api-access-m87fm" pod "32025ff2-9c7a-4a3c-92e9-db3462c61075" (UID: "32025ff2-9c7a-4a3c-92e9-db3462c61075"). InnerVolumeSpecName "kube-api-access-m87fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:41:53.575793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.575766    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-util" pod "32025ff2-9c7a-4a3c-92e9-db3462c61075" (UID: "32025ff2-9c7a-4a3c-92e9-db3462c61075"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:41:53.669755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.669667    4852 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-util\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:53.669755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.669697    4852 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32025ff2-9c7a-4a3c-92e9-db3462c61075-bundle\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:53.669755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:53.669707    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m87fm\" (UniqueName: \"kubernetes.io/projected/32025ff2-9c7a-4a3c-92e9-db3462c61075-kube-api-access-m87fm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:41:54.249657 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:54.249620    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:54.352283 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:54.352203    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:54.426046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:54.426011    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx" event={"ID":"32025ff2-9c7a-4a3c-92e9-db3462c61075","Type":"ContainerDied","Data":"aa16e80f82a735bc2df38d8a2b7829bdb9ee19b7377013a6fb39394547aa516d"}
Oct 30 20:41:54.426046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:54.426034    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644vdxrx"
Oct 30 20:41:54.426537 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:54.426094    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa16e80f82a735bc2df38d8a2b7829bdb9ee19b7377013a6fb39394547aa516d"
Oct 30 20:41:55.143795 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.143762    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:55.302060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302021    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"]
Oct 30 20:41:55.302356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302339    4852 cpu_manager.go:414] "containerMap: removing container" podUID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302359    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302377    4852 cpu_manager.go:414] "containerMap: removing container" podUID="1d081644-ab43-4c44-b830-f0f925404b17" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302399    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d081644-ab43-4c44-b830-f0f925404b17" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302409    4852 cpu_manager.go:414] "containerMap: removing container" podUID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302417    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302431    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302440    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302450    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b1802967-ee76-4b7c-af13-6921d7d59822" containerName="util"
Oct 30 20:41:55.302457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302457    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1802967-ee76-4b7c-af13-6921d7d59822" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302468    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302475    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302488    4852 cpu_manager.go:414] "containerMap: removing container" podUID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302495    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302504    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b1802967-ee76-4b7c-af13-6921d7d59822" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302511    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1802967-ee76-4b7c-af13-6921d7d59822" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302520    4852 cpu_manager.go:414] "containerMap: removing container" podUID="1d081644-ab43-4c44-b830-f0f925404b17" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302527    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d081644-ab43-4c44-b830-f0f925404b17" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302536    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302545    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302554    4852 cpu_manager.go:414] "containerMap: removing container" podUID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302561    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302571    4852 cpu_manager.go:414] "containerMap: removing container" podUID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302577    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302588    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302595    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302604    4852 cpu_manager.go:414] "containerMap: removing container" podUID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302611    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302620    4852 cpu_manager.go:414] "containerMap: removing container" podUID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302626    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302637    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302644    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302652    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302666    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302679    4852 cpu_manager.go:414] "containerMap: removing container" podUID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302686    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302697    4852 cpu_manager.go:414] "containerMap: removing container" podUID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302703    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302711    4852 cpu_manager.go:414] "containerMap: removing container" podUID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302717    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302732    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302739    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerName="util"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302752    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302758    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302768    4852 cpu_manager.go:414] "containerMap: removing container" podUID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302777    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302788    4852 cpu_manager.go:414] "containerMap: removing container" podUID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302794    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302804    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302810    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302824    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302832    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302847    4852 cpu_manager.go:414] "containerMap: removing container" podUID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302854    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302864    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b1802967-ee76-4b7c-af13-6921d7d59822" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302871    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1802967-ee76-4b7c-af13-6921d7d59822" containerName="extract"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302883    4852 cpu_manager.go:414] "containerMap: removing container" podUID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302889    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302898    4852 cpu_manager.go:414] "containerMap: removing container" podUID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302904    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302915    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302921    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302931    4852 cpu_manager.go:414] "containerMap: removing container" podUID="1d081644-ab43-4c44-b830-f0f925404b17" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302937    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d081644-ab43-4c44-b830-f0f925404b17" containerName="pull"
Oct 30 20:41:55.302914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302948    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerName="util"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.302955    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerName="util"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303045    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbd673b6-9681-421c-911c-0b2f9db7256b" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303059    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d081644-ab43-4c44-b830-f0f925404b17" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303069    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="70404f30-0e3d-40f3-9865-bb17b198d7e6" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303077    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d98b4f3b-e39c-4668-a8ca-1d06a83bed05" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303085    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="74738e1d-b3a7-4d9a-97a9-d8d1f437ea3b" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303095    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8e9895a-f3e9-49c7-94d1-920a0c81dcf7" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303103    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="32025ff2-9c7a-4a3c-92e9-db3462c61075" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303111    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="35042d17-85c3-401a-87e3-8fa7164f9c8d" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303119    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec28443b-4625-41fd-8a5d-caf060a2d34b" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303128    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1802967-ee76-4b7c-af13-6921d7d59822" containerName="extract"
Oct 30 20:41:55.305021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.303138    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b92104c8-4001-4003-80ae-7ad29ca336e0" containerName="extract"
Oct 30 20:41:55.305563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.305449    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.313925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.313904    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 20:41:55.314049 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.313990    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 20:41:55.314049 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.313993    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 20:41:55.314049 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.314040    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 20:41:55.315061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.315045    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-x2mvh\""
Oct 30 20:41:55.315161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.315075    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 20:41:55.382093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.382060    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-webhook-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.382236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.382097    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z28m4\" (UniqueName: \"kubernetes.io/projected/55058a52-53ff-45aa-9cda-946e6bc07d85-kube-api-access-z28m4\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.382236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.382129    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55058a52-53ff-45aa-9cda-946e6bc07d85-tmp\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.382318 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.382240    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-metrics-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.382318 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.382279    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.382413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.382380    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55058a52-53ff-45aa-9cda-946e6bc07d85-manifests\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.414614 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.414546    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"]
Oct 30 20:41:55.483579 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.483545    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-metrics-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.484018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.483587    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.484018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.483638    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55058a52-53ff-45aa-9cda-946e6bc07d85-manifests\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.484018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.483663    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-webhook-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.484018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.483683    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z28m4\" (UniqueName: \"kubernetes.io/projected/55058a52-53ff-45aa-9cda-946e6bc07d85-kube-api-access-z28m4\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.484018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.483714    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55058a52-53ff-45aa-9cda-946e6bc07d85-tmp\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.484286 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.484102    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/55058a52-53ff-45aa-9cda-946e6bc07d85-manifests\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.485962 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.485936    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55058a52-53ff-45aa-9cda-946e6bc07d85-tmp\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.486094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.486026    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-metrics-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.486171 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.486137    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-webhook-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.486298 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.486277    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/55058a52-53ff-45aa-9cda-946e6bc07d85-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.491134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.491110    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:55.493630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.493610    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z28m4\" (UniqueName: \"kubernetes.io/projected/55058a52-53ff-45aa-9cda-946e6bc07d85-kube-api-access-z28m4\") pod \"opendatahub-operator-controller-manager-597bf9dc6d-mxkdz\" (UID: \"55058a52-53ff-45aa-9cda-946e6bc07d85\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.618089 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.618053    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:41:55.624836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.624802    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:55.773610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.773574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:55.875403 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.875366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:55.929466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.929394    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"]
Oct 30 20:41:55.933906 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:41:55.933878    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55058a52_53ff_45aa_9cda_946e6bc07d85.slice/crio-5a7feeec1dfb3e9612d5ad17a43f410ab2ffa01014958ae7e6a812355b3455de WatchSource:0}: Error finding container 5a7feeec1dfb3e9612d5ad17a43f410ab2ffa01014958ae7e6a812355b3455de: Status 404 returned error can't find the container with id 5a7feeec1dfb3e9612d5ad17a43f410ab2ffa01014958ae7e6a812355b3455de
Oct 30 20:41:55.990207 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:55.990167    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:56.434997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:56.434964    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz" event={"ID":"55058a52-53ff-45aa-9cda-946e6bc07d85","Type":"ContainerStarted","Data":"5a7feeec1dfb3e9612d5ad17a43f410ab2ffa01014958ae7e6a812355b3455de"}
Oct 30 20:41:57.854959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:57.854920    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:57.955725 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:57.955682    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:41:59.451602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:59.451570    4852 generic.go:356] "Generic (PLEG): container finished" podID="55058a52-53ff-45aa-9cda-946e6bc07d85" containerID="d4e6255e564011052eff30b44dde50c385e0f815745e3f0a92cfc13f69c8c6ab" exitCode=0
Oct 30 20:41:59.452063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:59.451615    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz" event={"ID":"55058a52-53ff-45aa-9cda-946e6bc07d85","Type":"ContainerDied","Data":"d4e6255e564011052eff30b44dde50c385e0f815745e3f0a92cfc13f69c8c6ab"}
Oct 30 20:41:59.486825 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:41:59.486789    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:00.459514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:00.459472    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz" event={"ID":"55058a52-53ff-45aa-9cda-946e6bc07d85","Type":"ContainerStarted","Data":"0f3b2c551f177e7f86c60f9ce8ed0892f6887913611ff19c9edde6ecf488daf4"}
Oct 30 20:42:00.459947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:00.459602    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:42:00.512080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:00.512018    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz" podStartSLOduration=1.758478121 podStartE2EDuration="5.512002747s" totalImagesPullingTime="3.294156411s" totalInitContainerRuntime="459.368215ms" isStatefulPod=false podCreationTimestamp="2026-10-30 20:41:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:00.510299442 +0000 UTC m=+228.625033633" watchObservedRunningTime="2026-10-30 20:42:00.512002747 +0000 UTC m=+228.626736937"
Oct 30 20:42:02.694094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.694057    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"]
Oct 30 20:42:02.700834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.700816    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:02.710686 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.710655    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"]
Oct 30 20:42:02.712624 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.712606    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 20:42:02.712729 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.712717    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-hszk7\""
Oct 30 20:42:02.712851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.712836    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:42:02.747644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.747612    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1909b721-6370-4396-989b-a0d3e57d8ae3-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2\" (UID: \"1909b721-6370-4396-989b-a0d3e57d8ae3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:02.747764 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.747648    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d9ww\" (UniqueName: \"kubernetes.io/projected/1909b721-6370-4396-989b-a0d3e57d8ae3-kube-api-access-2d9ww\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2\" (UID: \"1909b721-6370-4396-989b-a0d3e57d8ae3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:02.830794 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.830753    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:02.848610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.848576    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1909b721-6370-4396-989b-a0d3e57d8ae3-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2\" (UID: \"1909b721-6370-4396-989b-a0d3e57d8ae3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:02.848748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.848620    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d9ww\" (UniqueName: \"kubernetes.io/projected/1909b721-6370-4396-989b-a0d3e57d8ae3-kube-api-access-2d9ww\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2\" (UID: \"1909b721-6370-4396-989b-a0d3e57d8ae3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:02.849001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.848983    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1909b721-6370-4396-989b-a0d3e57d8ae3-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2\" (UID: \"1909b721-6370-4396-989b-a0d3e57d8ae3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:02.866115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.866089    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d9ww\" (UniqueName: \"kubernetes.io/projected/1909b721-6370-4396-989b-a0d3e57d8ae3-kube-api-access-2d9ww\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2\" (UID: \"1909b721-6370-4396-989b-a0d3e57d8ae3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:02.935655 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:02.935622    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:03.012480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:03.012458    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"
Oct 30 20:42:03.299166 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:03.299013    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2"]
Oct 30 20:42:03.302177 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:03.302150    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1909b721_6370_4396_989b_a0d3e57d8ae3.slice/crio-70718fa3a6dfc49f5d2ee719189e7fb2955373f95f911ce878c116b174f5e287 WatchSource:0}: Error finding container 70718fa3a6dfc49f5d2ee719189e7fb2955373f95f911ce878c116b174f5e287: Status 404 returned error can't find the container with id 70718fa3a6dfc49f5d2ee719189e7fb2955373f95f911ce878c116b174f5e287
Oct 30 20:42:03.472336 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:03.472296    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2" event={"ID":"1909b721-6370-4396-989b-a0d3e57d8ae3","Type":"ContainerStarted","Data":"70718fa3a6dfc49f5d2ee719189e7fb2955373f95f911ce878c116b174f5e287"}
Oct 30 20:42:05.416270 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:05.416239    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:05.483608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:05.483563    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2" event={"ID":"1909b721-6370-4396-989b-a0d3e57d8ae3","Type":"ContainerStarted","Data":"d41eeed5b2e8cbcf2d0bbad65418547b0d85538c0596aea09f38c4d4e4ea4e86"}
Oct 30 20:42:05.506974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:05.506887    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-hr4w2" podStartSLOduration=1.403467701 podStartE2EDuration="3.506842348s" totalImagesPullingTime="2.103374647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:05.50461106 +0000 UTC m=+233.619345251" watchObservedRunningTime="2026-10-30 20:42:05.506842348 +0000 UTC m=+233.621576537"
Oct 30 20:42:05.787102 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:05.787073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:06.659885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:06.659846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:07.358086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:07.358047    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:07.491113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:07.491078    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:07.645880 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:07.645793    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:08.218476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:08.218445    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:08.328866 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:08.328821    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:09.731763 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:09.731729    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:09.949663 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:09.949626    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:11.004015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.003980    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"]
Oct 30 20:42:11.007199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.007182    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.016374 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.016348    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"]
Oct 30 20:42:11.017072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.017052    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 20:42:11.017747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.017732    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 20:42:11.017837 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.017816    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-w97pq\""
Oct 30 20:42:11.122055 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.122021    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed8a6ee0-7967-4691-a185-cbef0f21057a-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-7jcqr\" (UID: \"ed8a6ee0-7967-4691-a185-cbef0f21057a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.122238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.122086    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct5vf\" (UniqueName: \"kubernetes.io/projected/ed8a6ee0-7967-4691-a185-cbef0f21057a-kube-api-access-ct5vf\") pod \"cert-manager-cainjector-5c57bbd4fc-7jcqr\" (UID: \"ed8a6ee0-7967-4691-a185-cbef0f21057a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.223523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.223486    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed8a6ee0-7967-4691-a185-cbef0f21057a-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-7jcqr\" (UID: \"ed8a6ee0-7967-4691-a185-cbef0f21057a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.223684 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.223548    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct5vf\" (UniqueName: \"kubernetes.io/projected/ed8a6ee0-7967-4691-a185-cbef0f21057a-kube-api-access-ct5vf\") pod \"cert-manager-cainjector-5c57bbd4fc-7jcqr\" (UID: \"ed8a6ee0-7967-4691-a185-cbef0f21057a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.232161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.232127    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed8a6ee0-7967-4691-a185-cbef0f21057a-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-7jcqr\" (UID: \"ed8a6ee0-7967-4691-a185-cbef0f21057a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.232292 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.232198    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct5vf\" (UniqueName: \"kubernetes.io/projected/ed8a6ee0-7967-4691-a185-cbef0f21057a-kube-api-access-ct5vf\") pod \"cert-manager-cainjector-5c57bbd4fc-7jcqr\" (UID: \"ed8a6ee0-7967-4691-a185-cbef0f21057a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.330514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.330476    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"
Oct 30 20:42:11.465536 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.465504    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-597bf9dc6d-mxkdz"
Oct 30 20:42:11.611594 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:11.611571    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr"]
Oct 30 20:42:11.613936 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:11.613910    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded8a6ee0_7967_4691_a185_cbef0f21057a.slice/crio-ebe9e016e85f0ac7d2f7153bea446362eed8840910d913aebc8f97713a994580 WatchSource:0}: Error finding container ebe9e016e85f0ac7d2f7153bea446362eed8840910d913aebc8f97713a994580: Status 404 returned error can't find the container with id ebe9e016e85f0ac7d2f7153bea446362eed8840910d913aebc8f97713a994580
Oct 30 20:42:12.513746 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.513709    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr" event={"ID":"ed8a6ee0-7967-4691-a185-cbef0f21057a","Type":"ContainerStarted","Data":"ebe9e016e85f0ac7d2f7153bea446362eed8840910d913aebc8f97713a994580"}
Oct 30 20:42:12.613480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.613442    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"]
Oct 30 20:42:12.616559 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.616544    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:12.620603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.620577    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 20:42:12.621354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.621322    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9gn9q\""
Oct 30 20:42:12.621354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.621324    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 20:42:12.626978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.626956    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"]
Oct 30 20:42:12.735618 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.735586    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmk7l\" (UniqueName: \"kubernetes.io/projected/ea65a852-a9fb-4941-8c65-2de869589bbd-kube-api-access-bmk7l\") pod \"kuadrant-operator-controller-manager-cf8469d7b-tbbnz\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:12.735824 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.735661    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea65a852-a9fb-4941-8c65-2de869589bbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-tbbnz\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:12.836323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.836231    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmk7l\" (UniqueName: \"kubernetes.io/projected/ea65a852-a9fb-4941-8c65-2de869589bbd-kube-api-access-bmk7l\") pod \"kuadrant-operator-controller-manager-cf8469d7b-tbbnz\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:12.836515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.836320    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea65a852-a9fb-4941-8c65-2de869589bbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-tbbnz\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:12.836773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.836746    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea65a852-a9fb-4941-8c65-2de869589bbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-tbbnz\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:12.849159 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.849120    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmk7l\" (UniqueName: \"kubernetes.io/projected/ea65a852-a9fb-4941-8c65-2de869589bbd-kube-api-access-bmk7l\") pod \"kuadrant-operator-controller-manager-cf8469d7b-tbbnz\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:12.929343 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:12.929295    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:13.220542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:13.220463    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"]
Oct 30 20:42:13.224819 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:13.224779    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea65a852_a9fb_4941_8c65_2de869589bbd.slice/crio-b9917bcdd5c08bd3ca9c8e1909bc5e0b988a1d72472ef95b677dcabe88efd1f6 WatchSource:0}: Error finding container b9917bcdd5c08bd3ca9c8e1909bc5e0b988a1d72472ef95b677dcabe88efd1f6: Status 404 returned error can't find the container with id b9917bcdd5c08bd3ca9c8e1909bc5e0b988a1d72472ef95b677dcabe88efd1f6
Oct 30 20:42:13.520615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:13.520578    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz" event={"ID":"ea65a852-a9fb-4941-8c65-2de869589bbd","Type":"ContainerStarted","Data":"b9917bcdd5c08bd3ca9c8e1909bc5e0b988a1d72472ef95b677dcabe88efd1f6"}
Oct 30 20:42:14.258812 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:14.258769    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:14.900317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:14.900277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:15.532694 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:15.532652    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr" event={"ID":"ed8a6ee0-7967-4691-a185-cbef0f21057a","Type":"ContainerStarted","Data":"c017ef86507c9d187a4084f9230c6da78e1dace13845825c07f9f6259128bc96"}
Oct 30 20:42:15.555185 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:15.555121    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-7jcqr" podStartSLOduration=2.5087638439999997 podStartE2EDuration="5.55509901s" totalImagesPullingTime="3.046335166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:15.552737513 +0000 UTC m=+243.667471704" watchObservedRunningTime="2026-10-30 20:42:15.55509901 +0000 UTC m=+243.669833201"
Oct 30 20:42:15.966124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:15.966020    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:16.069353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:16.069318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:16.664813 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:16.664775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:16.766490 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:16.766456    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:17.278939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.278903    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"]
Oct 30 20:42:17.300236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.299419    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"]
Oct 30 20:42:17.300236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.299552    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"
Oct 30 20:42:17.310826 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.310803    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 20:42:17.312186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.312164    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-t8gtt\""
Oct 30 20:42:17.377063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.377028    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvjjc\" (UniqueName: \"kubernetes.io/projected/cc0ffb68-6f72-4b62-8069-da67c20f530a-kube-api-access-gvjjc\") pod \"dns-operator-controller-manager-8557cbdb84-dpr68\" (UID: \"cc0ffb68-6f72-4b62-8069-da67c20f530a\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"
Oct 30 20:42:17.478618 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.478578    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvjjc\" (UniqueName: \"kubernetes.io/projected/cc0ffb68-6f72-4b62-8069-da67c20f530a-kube-api-access-gvjjc\") pod \"dns-operator-controller-manager-8557cbdb84-dpr68\" (UID: \"cc0ffb68-6f72-4b62-8069-da67c20f530a\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"
Oct 30 20:42:17.486254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.486220    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:17.488951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.488924    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvjjc\" (UniqueName: \"kubernetes.io/projected/cc0ffb68-6f72-4b62-8069-da67c20f530a-kube-api-access-gvjjc\") pod \"dns-operator-controller-manager-8557cbdb84-dpr68\" (UID: \"cc0ffb68-6f72-4b62-8069-da67c20f530a\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"
Oct 30 20:42:17.634701 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.634615    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"
Oct 30 20:42:17.940090 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:17.940002    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"]
Oct 30 20:42:17.944146 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:17.944114    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0ffb68_6f72_4b62_8069_da67c20f530a.slice/crio-203e26146030d6e2b715ef17897cae69687d1347b8a4ec248d939304b991a245 WatchSource:0}: Error finding container 203e26146030d6e2b715ef17897cae69687d1347b8a4ec248d939304b991a245: Status 404 returned error can't find the container with id 203e26146030d6e2b715ef17897cae69687d1347b8a4ec248d939304b991a245
Oct 30 20:42:18.020105 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:18.020075    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:18.550172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:18.550143    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68" event={"ID":"cc0ffb68-6f72-4b62-8069-da67c20f530a","Type":"ContainerStarted","Data":"203e26146030d6e2b715ef17897cae69687d1347b8a4ec248d939304b991a245"}
Oct 30 20:42:18.784850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:18.784775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:18.899723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:18.899683    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:19.430702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.430661    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"]
Oct 30 20:42:19.432895 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.432875    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.443493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.443452    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 20:42:19.443763 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.443742    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 20:42:19.445421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.445395    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"]
Oct 30 20:42:19.528969 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.528939    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-dprts\""
Oct 30 20:42:19.563785 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.563743    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz" event={"ID":"ea65a852-a9fb-4941-8c65-2de869589bbd","Type":"ContainerStarted","Data":"f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e"}
Oct 30 20:42:19.564250 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.563835    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:19.597448 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.597413    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8cae76f-50cc-441d-9d0a-419466cdbccd-tmp\") pod \"jobset-operator-674cd59dc5-rdfxp\" (UID: \"b8cae76f-50cc-441d-9d0a-419466cdbccd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.597600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.597572    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnc9f\" (UniqueName: \"kubernetes.io/projected/b8cae76f-50cc-441d-9d0a-419466cdbccd-kube-api-access-fnc9f\") pod \"jobset-operator-674cd59dc5-rdfxp\" (UID: \"b8cae76f-50cc-441d-9d0a-419466cdbccd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.599439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.597661    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz" podStartSLOduration=2.302775733 podStartE2EDuration="7.597645365s" totalImagesPullingTime="5.294869632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:19.596185441 +0000 UTC m=+247.710919631" watchObservedRunningTime="2026-10-30 20:42:19.597645365 +0000 UTC m=+247.712379556"
Oct 30 20:42:19.698212 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.698122    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnc9f\" (UniqueName: \"kubernetes.io/projected/b8cae76f-50cc-441d-9d0a-419466cdbccd-kube-api-access-fnc9f\") pod \"jobset-operator-674cd59dc5-rdfxp\" (UID: \"b8cae76f-50cc-441d-9d0a-419466cdbccd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.698408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.698340    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8cae76f-50cc-441d-9d0a-419466cdbccd-tmp\") pod \"jobset-operator-674cd59dc5-rdfxp\" (UID: \"b8cae76f-50cc-441d-9d0a-419466cdbccd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.698772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.698747    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8cae76f-50cc-441d-9d0a-419466cdbccd-tmp\") pod \"jobset-operator-674cd59dc5-rdfxp\" (UID: \"b8cae76f-50cc-441d-9d0a-419466cdbccd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.708220 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.708191    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnc9f\" (UniqueName: \"kubernetes.io/projected/b8cae76f-50cc-441d-9d0a-419466cdbccd-kube-api-access-fnc9f\") pod \"jobset-operator-674cd59dc5-rdfxp\" (UID: \"b8cae76f-50cc-441d-9d0a-419466cdbccd\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.745115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.745070    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"
Oct 30 20:42:19.793968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.793931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:19.894921 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:19.894887    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:20.026099 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:20.026056    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp"]
Oct 30 20:42:20.029124 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:20.029088    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8cae76f_50cc_441d_9d0a_419466cdbccd.slice/crio-5821efee5e98205ea0bfb15d91d4879a9f65031c29694f795e4473212d589ca9 WatchSource:0}: Error finding container 5821efee5e98205ea0bfb15d91d4879a9f65031c29694f795e4473212d589ca9: Status 404 returned error can't find the container with id 5821efee5e98205ea0bfb15d91d4879a9f65031c29694f795e4473212d589ca9
Oct 30 20:42:20.570036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:20.569953    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp" event={"ID":"b8cae76f-50cc-441d-9d0a-419466cdbccd","Type":"ContainerStarted","Data":"5821efee5e98205ea0bfb15d91d4879a9f65031c29694f795e4473212d589ca9"}
Oct 30 20:42:20.571847 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:20.571817    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68" event={"ID":"cc0ffb68-6f72-4b62-8069-da67c20f530a","Type":"ContainerStarted","Data":"5d004fc383d5ea172609f8b15a310fc6f3ba1b237878153383b5f79a67017f94"}
Oct 30 20:42:20.597173 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:20.596987    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68" podStartSLOduration=1.229689054 podStartE2EDuration="3.596972713s" totalImagesPullingTime="2.367283659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:20.596343029 +0000 UTC m=+248.711077222" watchObservedRunningTime="2026-10-30 20:42:20.596972713 +0000 UTC m=+248.711706903"
Oct 30 20:42:21.288782 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:21.288750    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:21.363246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:21.363189    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:21.563275 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:42:21.563201    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:42:21.576943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:21.576912    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"
Oct 30 20:42:22.168987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.168955    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:22.277743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.277709    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:22.526833 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.526799    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"]
Oct 30 20:42:22.530437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.530417    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"
Oct 30 20:42:22.534493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.534471    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-m5q9s\""
Oct 30 20:42:22.546294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.546273    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"]
Oct 30 20:42:22.585258 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.585219    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp" event={"ID":"b8cae76f-50cc-441d-9d0a-419466cdbccd","Type":"ContainerStarted","Data":"42a68f7a1730d1f6915eca878225b48aaa6bdd93ad64ef5cbda704bfc5a70800"}
Oct 30 20:42:22.624849 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.624804    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6c8h\" (UniqueName: \"kubernetes.io/projected/45994274-c5d2-4041-8f70-7dc2304f1580-kube-api-access-m6c8h\") pod \"authorino-operator-7b7bb4bc77-dtgm8\" (UID: \"45994274-c5d2-4041-8f70-7dc2304f1580\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"
Oct 30 20:42:22.644527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.644476    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rdfxp" podStartSLOduration=1.5877707129999998 podStartE2EDuration="3.644461905s" totalImagesPullingTime="2.056691192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:22.642340669 +0000 UTC m=+250.757074861" watchObservedRunningTime="2026-10-30 20:42:22.644461905 +0000 UTC m=+250.759196092"
Oct 30 20:42:22.726006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.725965    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6c8h\" (UniqueName: \"kubernetes.io/projected/45994274-c5d2-4041-8f70-7dc2304f1580-kube-api-access-m6c8h\") pod \"authorino-operator-7b7bb4bc77-dtgm8\" (UID: \"45994274-c5d2-4041-8f70-7dc2304f1580\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"
Oct 30 20:42:22.740415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.740343    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6c8h\" (UniqueName: \"kubernetes.io/projected/45994274-c5d2-4041-8f70-7dc2304f1580-kube-api-access-m6c8h\") pod \"authorino-operator-7b7bb4bc77-dtgm8\" (UID: \"45994274-c5d2-4041-8f70-7dc2304f1580\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"
Oct 30 20:42:22.819271 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.819199    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:22.843619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.843585    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"
Oct 30 20:42:22.936325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.935282    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"]
Oct 30 20:42:22.942430 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.940285    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:22.946423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.946001    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-8rmkg\""
Oct 30 20:42:22.964071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.964046    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"]
Oct 30 20:42:22.990005 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:22.989971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:23.029538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029505    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0054779a-9980-4544-a064-233ea0e46755-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.029715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029548    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0054779a-9980-4544-a064-233ea0e46755-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.029715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029632    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.029715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029669    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0054779a-9980-4544-a064-233ea0e46755-istio-token\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.029715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029697    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.029929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029822    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.029929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029867    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqb97\" (UniqueName: \"kubernetes.io/projected/0054779a-9980-4544-a064-233ea0e46755-kube-api-access-pqb97\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.029929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029900    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-istio-data\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.030026 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.029968    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131008 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.130920    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0054779a-9980-4544-a064-233ea0e46755-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131008 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.130974    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131090    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0054779a-9980-4544-a064-233ea0e46755-istio-token\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131136    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131180    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131229    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqb97\" (UniqueName: \"kubernetes.io/projected/0054779a-9980-4544-a064-233ea0e46755-kube-api-access-pqb97\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131258    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-istio-data\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131295    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131374    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0054779a-9980-4544-a064-233ea0e46755-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131406    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131500    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131711    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.131851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.131831    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-istio-data\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.132248 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.132220    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0054779a-9980-4544-a064-233ea0e46755-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.133819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.133797    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0054779a-9980-4544-a064-233ea0e46755-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.133990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.133973    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0054779a-9980-4544-a064-233ea0e46755-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.141093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.141069    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:23.147929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.147901    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqb97\" (UniqueName: \"kubernetes.io/projected/0054779a-9980-4544-a064-233ea0e46755-kube-api-access-pqb97\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.159921 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.159895    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"]
Oct 30 20:42:23.161562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.161535    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0054779a-9980-4544-a064-233ea0e46755-istio-token\") pod \"data-science-gateway-data-science-gateway-class-57dc67465-62gx5\" (UID: \"0054779a-9980-4544-a064-233ea0e46755\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.162949 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:23.162927    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45994274_c5d2_4041_8f70_7dc2304f1580.slice/crio-ce62ea80d1fa5a44e19573949b1902e94d9af06b93b72e5a5d49d19b8183e094 WatchSource:0}: Error finding container ce62ea80d1fa5a44e19573949b1902e94d9af06b93b72e5a5d49d19b8183e094: Status 404 returned error can't find the container with id ce62ea80d1fa5a44e19573949b1902e94d9af06b93b72e5a5d49d19b8183e094
Oct 30 20:42:23.258532 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.258498    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:23.465482 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.463855    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:23.548192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.548158    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"]
Oct 30 20:42:23.551368 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:23.551337    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0054779a_9980_4544_a064_233ea0e46755.slice/crio-2d78230e3807c45d7531311ef2e471e997b14179cb61cd5c86253784b488f62a WatchSource:0}: Error finding container 2d78230e3807c45d7531311ef2e471e997b14179cb61cd5c86253784b488f62a: Status 404 returned error can't find the container with id 2d78230e3807c45d7531311ef2e471e997b14179cb61cd5c86253784b488f62a
Oct 30 20:42:23.553575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.553542    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:42:23.553663 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.553620    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:42:23.553663 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.553656    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:42:23.592169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.592089    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5" event={"ID":"0054779a-9980-4544-a064-233ea0e46755","Type":"ContainerStarted","Data":"2d78230e3807c45d7531311ef2e471e997b14179cb61cd5c86253784b488f62a"}
Oct 30 20:42:23.593617 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.593591    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8" event={"ID":"45994274-c5d2-4041-8f70-7dc2304f1580","Type":"ContainerStarted","Data":"ce62ea80d1fa5a44e19573949b1902e94d9af06b93b72e5a5d49d19b8183e094"}
Oct 30 20:42:23.785237 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:23.785202    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:24.391440 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.391172    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:24.519420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.519371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:24.600379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.600340    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5" event={"ID":"0054779a-9980-4544-a064-233ea0e46755","Type":"ContainerStarted","Data":"ebcda0b9a06daf1f13d308c59759a9529213c7f1c56e6722e42ed11e00934d60"}
Oct 30 20:42:24.629054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.629003    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5" podStartSLOduration=2.628988775 podStartE2EDuration="2.628988775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:24.626666463 +0000 UTC m=+252.741400652" watchObservedRunningTime="2026-10-30 20:42:24.628988775 +0000 UTC m=+252.743722965"
Oct 30 20:42:24.724467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.724355    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:24.878941 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.878909    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"]
Oct 30 20:42:24.883397 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.883363    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:24.894748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.894716    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-2mbgc\""
Oct 30 20:42:24.896562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.896537    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"]
Oct 30 20:42:24.902705 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.902680    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 20:42:24.902851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.902822    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 20:42:24.905504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:24.905483    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 20:42:25.052659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.052622    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22ztp\" (UniqueName: \"kubernetes.io/projected/f92a4844-d433-481b-a4cf-771108c24109-kube-api-access-22ztp\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.052659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.052660    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f92a4844-d433-481b-a4cf-771108c24109-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.052896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.052685    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f92a4844-d433-481b-a4cf-771108c24109-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.052896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.052744    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f92a4844-d433-481b-a4cf-771108c24109-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.154144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.154106    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22ztp\" (UniqueName: \"kubernetes.io/projected/f92a4844-d433-481b-a4cf-771108c24109-kube-api-access-22ztp\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.154144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.154146    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f92a4844-d433-481b-a4cf-771108c24109-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.154426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.154172    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f92a4844-d433-481b-a4cf-771108c24109-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.154426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.154187    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f92a4844-d433-481b-a4cf-771108c24109-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.154562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.154539    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f92a4844-d433-481b-a4cf-771108c24109-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.154637 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.154615    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f92a4844-d433-481b-a4cf-771108c24109-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.155273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.155252    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f92a4844-d433-481b-a4cf-771108c24109-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.170638 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.170606    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22ztp\" (UniqueName: \"kubernetes.io/projected/f92a4844-d433-481b-a4cf-771108c24109-kube-api-access-22ztp\") pod \"kserve-module-controller-manager-7566cf5fc8-jclms\" (UID: \"f92a4844-d433-481b-a4cf-771108c24109\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.197363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.197333    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:25.259325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.259294    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:25.264073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.264052    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:25.476789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.476764    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"]
Oct 30 20:42:25.479202 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:25.479169    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf92a4844_d433_481b_a4cf_771108c24109.slice/crio-c0290dae12c887bdfdd11f20e5280d962ea597bc25a188207adb08b7f7e17934 WatchSource:0}: Error finding container c0290dae12c887bdfdd11f20e5280d962ea597bc25a188207adb08b7f7e17934: Status 404 returned error can't find the container with id c0290dae12c887bdfdd11f20e5280d962ea597bc25a188207adb08b7f7e17934
Oct 30 20:42:25.570154 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.570111    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:25.612040 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.612009    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms" event={"ID":"f92a4844-d433-481b-a4cf-771108c24109","Type":"ContainerStarted","Data":"c0290dae12c887bdfdd11f20e5280d962ea597bc25a188207adb08b7f7e17934"}
Oct 30 20:42:25.612475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.612220    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:25.613090 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:25.613073    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-57dc67465-62gx5"
Oct 30 20:42:26.196335 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.195072    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"]
Oct 30 20:42:26.198937 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.198912    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"
Oct 30 20:42:26.204612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.204588    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5nvwk\""
Oct 30 20:42:26.211557 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.211488    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"]
Oct 30 20:42:26.246736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.246695    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:26.352818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.350549    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:26.367526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.367485    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b89h6\" (UniqueName: \"kubernetes.io/projected/43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4-kube-api-access-b89h6\") pod \"limitador-operator-controller-manager-5778bb4449-dwfh7\" (UID: \"43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"
Oct 30 20:42:26.468494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.468378    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b89h6\" (UniqueName: \"kubernetes.io/projected/43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4-kube-api-access-b89h6\") pod \"limitador-operator-controller-manager-5778bb4449-dwfh7\" (UID: \"43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"
Oct 30 20:42:26.482411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.482338    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b89h6\" (UniqueName: \"kubernetes.io/projected/43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4-kube-api-access-b89h6\") pod \"limitador-operator-controller-manager-5778bb4449-dwfh7\" (UID: \"43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"
Oct 30 20:42:26.521423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.517038    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"
Oct 30 20:42:26.829471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:26.829369    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"]
Oct 30 20:42:26.832371 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:26.832339    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43f6af0c_cf27_4018_ba25_fbbdbf7f7eb4.slice/crio-c4ef5e4eb5e5a92e74900cc96c9e654887c7096c37d38e4b9459569e178f5186 WatchSource:0}: Error finding container c4ef5e4eb5e5a92e74900cc96c9e654887c7096c37d38e4b9459569e178f5186: Status 404 returned error can't find the container with id c4ef5e4eb5e5a92e74900cc96c9e654887c7096c37d38e4b9459569e178f5186
Oct 30 20:42:27.037191 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:27.037125    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:27.626330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:27.626297    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7" event={"ID":"43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4","Type":"ContainerStarted","Data":"c4ef5e4eb5e5a92e74900cc96c9e654887c7096c37d38e4b9459569e178f5186"}
Oct 30 20:42:27.628275 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:27.628247    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8" event={"ID":"45994274-c5d2-4041-8f70-7dc2304f1580","Type":"ContainerStarted","Data":"3d0a90af7f09b499b6119523bb3d7992d1c50f794ad44319547980387ec93e1c"}
Oct 30 20:42:27.628466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:27.628447    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"
Oct 30 20:42:27.649870 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:27.649818    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8" podStartSLOduration=1.568873617 podStartE2EDuration="5.649801729s" totalImagesPullingTime="4.080928112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:27.648871928 +0000 UTC m=+255.763606119" watchObservedRunningTime="2026-10-30 20:42:27.649801729 +0000 UTC m=+255.764535919"
Oct 30 20:42:28.636662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:28.636622    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms" event={"ID":"f92a4844-d433-481b-a4cf-771108c24109","Type":"ContainerStarted","Data":"efdceef4d91f87c89327213f0fe9c746bbc0a81cb160732d072ad6c2089e7d66"}
Oct 30 20:42:28.637096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:28.636749    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:28.658532 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:28.658483    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms" podStartSLOduration=2.0858225519999998 podStartE2EDuration="4.65846686s" totalImagesPullingTime="2.572644308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:28.656429704 +0000 UTC m=+256.771163896" watchObservedRunningTime="2026-10-30 20:42:28.65846686 +0000 UTC m=+256.773201050"
Oct 30 20:42:29.643565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:29.643524    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7" event={"ID":"43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4","Type":"ContainerStarted","Data":"065ad2d0355b2f6df13af7351d124b2e96fce3624495ce4e45938ba8d530e280"}
Oct 30 20:42:29.661995 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:29.661947    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7" podStartSLOduration=1.606816762 podStartE2EDuration="3.66193433s" totalImagesPullingTime="2.055117568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:29.659654407 +0000 UTC m=+257.774388610" watchObservedRunningTime="2026-10-30 20:42:29.66193433 +0000 UTC m=+257.776668572"
Oct 30 20:42:29.827650 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:29.827613    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:29.910068 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:29.909966    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:30.430352 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:30.430306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:30.518396 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:30.518347    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:30.573682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:30.573655    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:30.641854 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:30.641822    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:30.648653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:30.648624    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"
Oct 30 20:42:30.740430 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:30.740324    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:32.588352 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:32.588324    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dpr68"
Oct 30 20:42:36.754046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:36.754013    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:36.843270 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:36.843235    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:38.639139 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:38.639110    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-dtgm8"
Oct 30 20:42:41.655423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:41.655369    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-dwfh7"
Oct 30 20:42:43.059167 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:43.059132    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:43.159311 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:43.159276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:46.076701 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:46.076666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:46.237016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:46.236977    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:47.657070 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.657037    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"]
Oct 30 20:42:47.657517 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.657296    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz" podUID="ea65a852-a9fb-4941-8c65-2de869589bbd" containerName="manager" containerID="cri-o://f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e" gracePeriod=10
Oct 30 20:42:47.784924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.784895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:47.895871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.895844    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:47.929514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.929445    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:47.957115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.957077    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea65a852-a9fb-4941-8c65-2de869589bbd-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea65a852-a9fb-4941-8c65-2de869589bbd-extensions-socket-volume\") pod \"ea65a852-a9fb-4941-8c65-2de869589bbd\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") "
Oct 30 20:42:47.957345 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.957137    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea65a852-a9fb-4941-8c65-2de869589bbd-kube-api-access-bmk7l\" (UniqueName: \"kubernetes.io/projected/ea65a852-a9fb-4941-8c65-2de869589bbd-kube-api-access-bmk7l\") pod \"ea65a852-a9fb-4941-8c65-2de869589bbd\" (UID: \"ea65a852-a9fb-4941-8c65-2de869589bbd\") "
Oct 30 20:42:47.957694 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.957666    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea65a852-a9fb-4941-8c65-2de869589bbd-extensions-socket-volume" pod "ea65a852-a9fb-4941-8c65-2de869589bbd" (UID: "ea65a852-a9fb-4941-8c65-2de869589bbd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:42:47.959710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:47.959680    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea65a852-a9fb-4941-8c65-2de869589bbd-kube-api-access-bmk7l" pod "ea65a852-a9fb-4941-8c65-2de869589bbd" (UID: "ea65a852-a9fb-4941-8c65-2de869589bbd"). InnerVolumeSpecName "kube-api-access-bmk7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:42:48.057907 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.057867    4852 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea65a852-a9fb-4941-8c65-2de869589bbd-extensions-socket-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:42:48.057907 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.057908    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmk7l\" (UniqueName: \"kubernetes.io/projected/ea65a852-a9fb-4941-8c65-2de869589bbd-kube-api-access-bmk7l\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:42:48.737755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.737722    4852 generic.go:356] "Generic (PLEG): container finished" podID="ea65a852-a9fb-4941-8c65-2de869589bbd" containerID="f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e" exitCode=0
Oct 30 20:42:48.738244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.737786    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"
Oct 30 20:42:48.738244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.737801    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz" event={"ID":"ea65a852-a9fb-4941-8c65-2de869589bbd","Type":"ContainerDied","Data":"f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e"}
Oct 30 20:42:48.738244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.737871    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz" event={"ID":"ea65a852-a9fb-4941-8c65-2de869589bbd","Type":"ContainerDied","Data":"b9917bcdd5c08bd3ca9c8e1909bc5e0b988a1d72472ef95b677dcabe88efd1f6"}
Oct 30 20:42:48.738244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.737906    4852 scope.go:122] "RemoveContainer" containerID="f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e"
Oct 30 20:42:48.746715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.746697    4852 scope.go:122] "RemoveContainer" containerID="f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e"
Oct 30 20:42:48.747015 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:42:48.746989    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e\": container with ID starting with f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e not found: ID does not exist" containerID="f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e"
Oct 30 20:42:48.747066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.747019    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e"} err="failed to get container status \"f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e\": rpc error: code = NotFound desc = could not find container \"f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e\": container with ID starting with f77dae3ccf92aaf91314af0ee60d3015f83cdf090464508d478f5c3d8404280e not found: ID does not exist"
Oct 30 20:42:48.788719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.788684    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"]
Oct 30 20:42:48.801652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:48.801618    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-tbbnz"]
Oct 30 20:42:49.645278 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.645249    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-jclms"
Oct 30 20:42:49.664449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.664412    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"]
Oct 30 20:42:49.664789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.664775    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ea65a852-a9fb-4941-8c65-2de869589bbd" containerName="manager"
Oct 30 20:42:49.664826 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.664791    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea65a852-a9fb-4941-8c65-2de869589bbd" containerName="manager"
Oct 30 20:42:49.664868 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.664859    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea65a852-a9fb-4941-8c65-2de869589bbd" containerName="manager"
Oct 30 20:42:49.699855 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.699823    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"]
Oct 30 20:42:49.700076 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.699931    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.708752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.708723    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 30 20:42:49.708752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.708748    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 30 20:42:49.708993 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.708819    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 30 20:42:49.708993 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.708902    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-l7ltr\""
Oct 30 20:42:49.772496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.772456    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-metrics-certs\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.772952 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.772551    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-cert\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.772952 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.772632    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hvvx\" (UniqueName: \"kubernetes.io/projected/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-kube-api-access-2hvvx\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.772952 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.772696    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-manager-config\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.873958 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.873915    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-metrics-certs\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.874137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.873997    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-cert\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.874137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.874031    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hvvx\" (UniqueName: \"kubernetes.io/projected/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-kube-api-access-2hvvx\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.874137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.874081    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-manager-config\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.874811 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.874779    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-manager-config\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.876503 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.876481    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-cert\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.876555 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.876539    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-metrics-certs\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:49.884616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:49.884589    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hvvx\" (UniqueName: \"kubernetes.io/projected/c6202ed5-3b85-4405-a5c9-37fc4cdc12f8-kube-api-access-2hvvx\") pod \"jobset-controller-manager-57fd4644b5-9vmrb\" (UID: \"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8\") " pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:50.003116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:50.003027    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea65a852-a9fb-4941-8c65-2de869589bbd" path="/var/lib/kubelet/pods/ea65a852-a9fb-4941-8c65-2de869589bbd/volumes"
Oct 30 20:42:50.024580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:50.024548    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:50.302930 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:50.302898    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"]
Oct 30 20:42:50.304863 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:42:50.304837    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6202ed5_3b85_4405_a5c9_37fc4cdc12f8.slice/crio-3501eecea4f7d1cab97bd9f8410dbc87ec5a2c5e2950ea3d97639952550c880f WatchSource:0}: Error finding container 3501eecea4f7d1cab97bd9f8410dbc87ec5a2c5e2950ea3d97639952550c880f: Status 404 returned error can't find the container with id 3501eecea4f7d1cab97bd9f8410dbc87ec5a2c5e2950ea3d97639952550c880f
Oct 30 20:42:50.752496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:50.752372    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb" event={"ID":"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8","Type":"ContainerStarted","Data":"3501eecea4f7d1cab97bd9f8410dbc87ec5a2c5e2950ea3d97639952550c880f"}
Oct 30 20:42:51.514268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:51.514214    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:51.556740 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:42:51.556702    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:42:51.612175 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:51.612141    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:53.770686 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:53.770646    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb" event={"ID":"c6202ed5-3b85-4405-a5c9-37fc4cdc12f8","Type":"ContainerStarted","Data":"71d19e26a3b4bba468cf75b8a7277d20d012bf3570e68ff5c864bdc88a925e95"}
Oct 30 20:42:53.771202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:53.770752    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:42:53.788508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:53.788453    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb" podStartSLOduration=1.672946308 podStartE2EDuration="4.788435442s" totalImagesPullingTime="3.115489134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:42:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:42:53.787405134 +0000 UTC m=+281.902139315" watchObservedRunningTime="2026-10-30 20:42:53.788435442 +0000 UTC m=+281.903169644"
Oct 30 20:42:57.984088 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:57.984019    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:42:58.086698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:42:58.086655    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:02.852094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:02.852059    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:04.544088 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:04.544050    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:04.644363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:04.644324    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:04.781737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:04.781706    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-57fd4644b5-9vmrb"
Oct 30 20:43:05.960450 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:05.960415    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:06.074696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:06.074647    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:06.147181 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:06.147145    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:06.961361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:06.961317    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:07.063030 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:07.062991    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:07.528696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:07.528659    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:07.629078 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:07.629042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:12.765815 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:12.765738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:12.871462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:12.871428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:13.267995 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:13.267950    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:13.386591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:13.386554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:13.480978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:13.480945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:18.706678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:18.706644    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:18.808730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:18.808688    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:18.931653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:18.931623    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:19.077720 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:19.077688    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:19.182092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:19.182054    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:19.654428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:19.654382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:19.869490 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:19.869431    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:20.325436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:20.325405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:20.430063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:20.430028    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:20.525661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:20.525634    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:20.627605 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:20.627518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:21.557111 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:43:21.557073    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:43:26.041553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:26.041491    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:26.335366 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:26.335284    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:27.118506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:27.118468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:29.007644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:29.007612    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:29.133595 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:29.133538    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:45.942332 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:45.942300    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:46.025161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:46.025126    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:46.108806 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:46.108772    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:46.208687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:46.208612    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:46.283719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:46.283661    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:51.556526 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:43:51.556490    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:43:52.304350 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:52.304315    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:52.416308 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:52.416277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:53.923702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:53.923664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:43:54.099116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:43:54.099080    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:02.958238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:02.958196    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:21.556969 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:44:21.556928    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:44:24.231114 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:24.231071    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:24.371097 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:24.371056    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:25.526583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:25.526531    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:25.657357 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:25.657315    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:25.772808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:25.772772    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:32.580833 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:32.580796    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:35.916201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:35.916104    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body=
Oct 30 20:44:35.916201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:35.916175    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused"
Oct 30 20:44:36.302255 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.302219    4852 generic.go:356] "Generic (PLEG): container finished" podID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerID="f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4" exitCode=0
Oct 30 20:44:36.302446 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.302290    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" event={"ID":"64c9e9e4-3f97-4165-b06e-b15c24cd9eba","Type":"ContainerDied","Data":"f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4"}
Oct 30 20:44:36.302712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.302695    4852 scope.go:122] "RemoveContainer" containerID="f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4"
Oct 30 20:44:36.303822 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.303794    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:44:36.303877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.303864    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:44:36.303912 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.303895    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:44:36.775925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.775873    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:36.967410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:36.966469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:37.309998 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:37.309958    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" event={"ID":"64c9e9e4-3f97-4165-b06e-b15c24cd9eba","Type":"ContainerStarted","Data":"af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5"}
Oct 30 20:44:37.830609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:37.830565    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:44:37.830863 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:37.830837    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body=
Oct 30 20:44:37.830954 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:37.830899    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused"
Oct 30 20:44:38.830342 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:38.830302    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body=
Oct 30 20:44:38.830773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:38.830363    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused"
Oct 30 20:44:39.830572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:39.830533    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body=
Oct 30 20:44:39.830966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:39.830592    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused"
Oct 30 20:44:40.831039 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:40.830998    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body=
Oct 30 20:44:40.831459 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:40.831068    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused"
Oct 30 20:44:41.831336 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:41.831295    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body=
Oct 30 20:44:41.831789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:41.831363    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused"
Oct 30 20:44:42.835012 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:42.834922    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:44:42.835536 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:42.835515    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:44:42.836156 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:42.836134    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 20:44:45.076950 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:44:45.076906    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:44:51.557102 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:44:51.557066    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:45:02.975821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:02.975784    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:14.805283 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:14.805243    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:14.954740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:14.954696    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:20.948439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:20.948405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:21.047297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:21.047265    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:21.556612 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:45:21.556577    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:45:45.469790 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:45.469757    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:45.986561 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:45.986520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:47.359330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:47.359297    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:47.471786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:47.471747    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:51.047595 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:51.047557    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:51.555680 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:45:51.555647    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:45:53.375358 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.375326    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"]
Oct 30 20:45:53.378818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.378793    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.387597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.387567    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 20:45:53.387747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.387669    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 20:45:53.387820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.387758    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s86lq\""
Oct 30 20:45:53.387820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.387788    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 20:45:53.388802 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.388778    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"]
Oct 30 20:45:53.415024 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.414990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:53.508853 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.508811    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjj6\" (UniqueName: \"kubernetes.io/projected/a0cd0d78-ca07-499a-b928-3ce2b1d00627-kube-api-access-pkjj6\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.508853 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.508870    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.509201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.509010    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.509201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.509066    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.509201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.509185    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd0d78-ca07-499a-b928-3ce2b1d00627-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.509361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.509319    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.610525 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.610482    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd0d78-ca07-499a-b928-3ce2b1d00627-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.610744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.610553    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.610744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.610585    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjj6\" (UniqueName: \"kubernetes.io/projected/a0cd0d78-ca07-499a-b928-3ce2b1d00627-kube-api-access-pkjj6\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.610744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.610607    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.610744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.610623    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.610744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.610641    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.611137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.611107    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.611233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.611139    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.611233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.611169    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.612943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.612911    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd0d78-ca07-499a-b928-3ce2b1d00627-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.613120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.613098    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd0d78-ca07-499a-b928-3ce2b1d00627-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.620907 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.620885    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjj6\" (UniqueName: \"kubernetes.io/projected/a0cd0d78-ca07-499a-b928-3ce2b1d00627-kube-api-access-pkjj6\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-hl2cc\" (UID: \"a0cd0d78-ca07-499a-b928-3ce2b1d00627\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:53.692735 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:53.692652    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:54.013170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:54.013148    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"]
Oct 30 20:45:54.015274 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:45:54.015248    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0cd0d78_ca07_499a_b928_3ce2b1d00627.slice/crio-daa0d43010fa41090acd51f1ea6d8fbf744692dc6c224429627162cc0fe93205 WatchSource:0}: Error finding container daa0d43010fa41090acd51f1ea6d8fbf744692dc6c224429627162cc0fe93205: Status 404 returned error can't find the container with id daa0d43010fa41090acd51f1ea6d8fbf744692dc6c224429627162cc0fe93205
Oct 30 20:45:54.016464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:54.016444    4852 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:45:54.703002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:54.702960    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc" event={"ID":"a0cd0d78-ca07-499a-b928-3ce2b1d00627","Type":"ContainerStarted","Data":"daa0d43010fa41090acd51f1ea6d8fbf744692dc6c224429627162cc0fe93205"}
Oct 30 20:45:55.385335 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:55.385303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:55.709504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:55.709415    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc" event={"ID":"a0cd0d78-ca07-499a-b928-3ce2b1d00627","Type":"ContainerStarted","Data":"aef77bef9e51f00aea4ea5f8130e720eb7ef0fd46d3549396ff813ba174b32b3"}
Oct 30 20:45:55.738950 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:55.738904    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc" podStartSLOduration=1.496619797 podStartE2EDuration="2.738890119s" totalImagesPullingTime="1.242270322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:45:55.735734241 +0000 UTC m=+463.850468431" watchObservedRunningTime="2026-10-30 20:45:55.738890119 +0000 UTC m=+463.853624329"
Oct 30 20:45:56.693807 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:56.693769    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:45:59.733905 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.733868    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"]
Oct 30 20:45:59.737551 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.737526    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.750462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.750433    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"]
Oct 30 20:45:59.762038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.761998    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.762203 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.762044    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7f9d76-006d-4453-a291-4037dd19730d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.762203 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.762144    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.762203 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.762190    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhvw9\" (UniqueName: \"kubernetes.io/projected/3f7f9d76-006d-4453-a291-4037dd19730d-kube-api-access-bhvw9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.762506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.762252    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.762506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.762293    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.812333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.812297    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 20:45:59.814199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.814176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:45:59.862990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.862953    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.862999    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7f9d76-006d-4453-a291-4037dd19730d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.863037    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.863064    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhvw9\" (UniqueName: \"kubernetes.io/projected/3f7f9d76-006d-4453-a291-4037dd19730d-kube-api-access-bhvw9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.863104    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.863126    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863533 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.863509    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.863567    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.863615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.863597    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.865262 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.865239    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f7f9d76-006d-4453-a291-4037dd19730d-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.865800 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.865781    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f7f9d76-006d-4453-a291-4037dd19730d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:45:59.872466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:45:59.872440    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhvw9\" (UniqueName: \"kubernetes.io/projected/3f7f9d76-006d-4453-a291-4037dd19730d-kube-api-access-bhvw9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-x27k5\" (UID: \"3f7f9d76-006d-4453-a291-4037dd19730d\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:46:00.052539 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.052501    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:46:00.333194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.333162    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"]
Oct 30 20:46:00.335308 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:00.335281    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7f9d76_006d_4453_a291_4037dd19730d.slice/crio-9d0e6961c922f2f367c7adc2927a2e70970f56e90e200b98ebd09245b1e6a929 WatchSource:0}: Error finding container 9d0e6961c922f2f367c7adc2927a2e70970f56e90e200b98ebd09245b1e6a929: Status 404 returned error can't find the container with id 9d0e6961c922f2f367c7adc2927a2e70970f56e90e200b98ebd09245b1e6a929
Oct 30 20:46:00.718755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.718712    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"]
Oct 30 20:46:00.722888 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.722862    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.741203 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.741170    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5" event={"ID":"3f7f9d76-006d-4453-a291-4037dd19730d","Type":"ContainerStarted","Data":"7e02179ede39205452e857b7e277d30c5c2583930134abce0f0380fb7b500d1d"}
Oct 30 20:46:00.741656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.741223    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5" event={"ID":"3f7f9d76-006d-4453-a291-4037dd19730d","Type":"ContainerStarted","Data":"9d0e6961c922f2f367c7adc2927a2e70970f56e90e200b98ebd09245b1e6a929"}
Oct 30 20:46:00.743419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.743379    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 20:46:00.745378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.745351    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"]
Oct 30 20:46:00.768772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.768732    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.768954 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.768778    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.768954 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.768831    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.769035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.768948    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc074dfb-fc56-4d06-aec4-ba0da629998e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.769035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.769003    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtqp8\" (UniqueName: \"kubernetes.io/projected/bc074dfb-fc56-4d06-aec4-ba0da629998e-kube-api-access-gtqp8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.769035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.769028    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.801154 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.801091    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:00.809711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.809652    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5" podStartSLOduration=1.495813493 podStartE2EDuration="1.809635948s" totalImagesPullingTime="313.822455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:45:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:46:00.808784828 +0000 UTC m=+468.923519019" watchObservedRunningTime="2026-10-30 20:46:00.809635948 +0000 UTC m=+468.924370141"
Oct 30 20:46:00.870502 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.870466    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.870661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.870529    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc074dfb-fc56-4d06-aec4-ba0da629998e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.870661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.870592    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtqp8\" (UniqueName: \"kubernetes.io/projected/bc074dfb-fc56-4d06-aec4-ba0da629998e-kube-api-access-gtqp8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.870661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.870628    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.870808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.870697    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.870808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.870733    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.870916 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.870894    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.871092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.871056    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.871213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.871138    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.873169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.873147    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc074dfb-fc56-4d06-aec4-ba0da629998e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.873480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.873461    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc074dfb-fc56-4d06-aec4-ba0da629998e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:00.879822 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:00.879792    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtqp8\" (UniqueName: \"kubernetes.io/projected/bc074dfb-fc56-4d06-aec4-ba0da629998e-kube-api-access-gtqp8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd\" (UID: \"bc074dfb-fc56-4d06-aec4-ba0da629998e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:01.037218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:01.037177    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:01.053518 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:01.053491    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:46:01.320947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:01.320922    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"]
Oct 30 20:46:01.324325 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:01.324296    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc074dfb_fc56_4d06_aec4_ba0da629998e.slice/crio-d362a87748e55b9306e95669522ebe378311e496a20fe97e38b870e6284b7c1e WatchSource:0}: Error finding container d362a87748e55b9306e95669522ebe378311e496a20fe97e38b870e6284b7c1e: Status 404 returned error can't find the container with id d362a87748e55b9306e95669522ebe378311e496a20fe97e38b870e6284b7c1e
Oct 30 20:46:01.748916 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:01.748881    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd" event={"ID":"bc074dfb-fc56-4d06-aec4-ba0da629998e","Type":"ContainerStarted","Data":"995e94a04a88acdd5301c0fdc6ee056bbc302f654affc0b60e223c3ec9b397b5"}
Oct 30 20:46:01.749426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:01.748932    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd" event={"ID":"bc074dfb-fc56-4d06-aec4-ba0da629998e","Type":"ContainerStarted","Data":"d362a87748e55b9306e95669522ebe378311e496a20fe97e38b870e6284b7c1e"}
Oct 30 20:46:01.772898 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:01.772827    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd" podStartSLOduration=1.472468602 podStartE2EDuration="1.772808196s" totalImagesPullingTime="300.339594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:46:01.770022115 +0000 UTC m=+469.884756305" watchObservedRunningTime="2026-10-30 20:46:01.772808196 +0000 UTC m=+469.887542388"
Oct 30 20:46:02.038291 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:02.038254    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:02.172221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:02.172185    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:03.095608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:03.095563    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:03.532502 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:03.532469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:03.693493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:03.693456    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:46:03.706075 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:03.706044    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:46:03.771800 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:03.771771    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-hl2cc"
Oct 30 20:46:04.388722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.388658    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"]
Oct 30 20:46:04.392319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.392297    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.401667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.400451    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 20:46:04.403340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.403293    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.403340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.403341    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.403584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.403422    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzxkc\" (UniqueName: \"kubernetes.io/projected/06b775dd-29f6-409e-8725-a476c704e31c-kube-api-access-tzxkc\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.403584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.403492    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.403584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.403551    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.403770 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.403583    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b775dd-29f6-409e-8725-a476c704e31c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.403947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.403922    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"]
Oct 30 20:46:04.463774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.463730    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:04.504375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504326    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzxkc\" (UniqueName: \"kubernetes.io/projected/06b775dd-29f6-409e-8725-a476c704e31c-kube-api-access-tzxkc\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.504590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504442    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.504590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504500    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.504590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504528    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b775dd-29f6-409e-8725-a476c704e31c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.504590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504559    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.504819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504594    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.504989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504965    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.505076 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.504988    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.505257 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.505234    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.507444 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.507419    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06b775dd-29f6-409e-8725-a476c704e31c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.507844 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.507822    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06b775dd-29f6-409e-8725-a476c704e31c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.515783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.515761    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzxkc\" (UniqueName: \"kubernetes.io/projected/06b775dd-29f6-409e-8725-a476c704e31c-kube-api-access-tzxkc\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5\" (UID: \"06b775dd-29f6-409e-8725-a476c704e31c\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.706323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.706168    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:04.990230 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:04.990199    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"]
Oct 30 20:46:04.992716 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:04.992684    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06b775dd_29f6_409e_8725_a476c704e31c.slice/crio-7fd639b08b3ec05ae35fbabceec75311b94e45885b3821754ffa63b389aec3c5 WatchSource:0}: Error finding container 7fd639b08b3ec05ae35fbabceec75311b94e45885b3821754ffa63b389aec3c5: Status 404 returned error can't find the container with id 7fd639b08b3ec05ae35fbabceec75311b94e45885b3821754ffa63b389aec3c5
Oct 30 20:46:05.783547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:05.783514    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" event={"ID":"06b775dd-29f6-409e-8725-a476c704e31c","Type":"ContainerStarted","Data":"c09cc9fe2e41a35251013e2720c70a091121621add91381e1e49c64a3394c7b8"}
Oct 30 20:46:05.783944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:05.783555    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" event={"ID":"06b775dd-29f6-409e-8725-a476c704e31c","Type":"ContainerStarted","Data":"7fd639b08b3ec05ae35fbabceec75311b94e45885b3821754ffa63b389aec3c5"}
Oct 30 20:46:05.810572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:05.810529    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podStartSLOduration=1.496535106 podStartE2EDuration="1.810516295s" totalImagesPullingTime="313.981189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:46:05.810321687 +0000 UTC m=+473.925055876" watchObservedRunningTime="2026-10-30 20:46:05.810516295 +0000 UTC m=+473.925250484"
Oct 30 20:46:06.707060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:06.707020    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:06.994396 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:06.994302    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"]
Oct 30 20:46:06.998540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:06.998503    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.018201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.018164    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"]
Oct 30 20:46:07.023945 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.023913    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.023958    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.024027    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.024050    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.024066    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024276 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.024099    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024276 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.024259    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.024281    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.024346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.024297    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47mb7\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-kube-api-access-47mb7\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125055 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125015    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125055 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125064    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125087    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125107    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125123    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125168    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125197    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125222    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125247    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47mb7\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-kube-api-access-47mb7\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125528    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125580    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125792    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.125972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125952    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.126014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.125992    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.127534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.127511    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.127785 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.127766    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.133980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.133958    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.134121 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.134061    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47mb7\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-kube-api-access-47mb7\") pod \"maas-default-gateway-openshift-default-77f556d647-7pzkt\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.320529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.320489    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:07.611253 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.611215    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"]
Oct 30 20:46:07.611758 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:07.611717    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f503e52_4e3f_42c4_a3bf_ed99f439a3a7.slice/crio-e4c8483e4bd3a7de4287da4593edf5f98b4934c230e51e0ac0d708159298f82b WatchSource:0}: Error finding container e4c8483e4bd3a7de4287da4593edf5f98b4934c230e51e0ac0d708159298f82b: Status 404 returned error can't find the container with id e4c8483e4bd3a7de4287da4593edf5f98b4934c230e51e0ac0d708159298f82b
Oct 30 20:46:07.614380 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.614349    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:07.614512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.614445    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:07.614512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.614476    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:07.797289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.797248    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" event={"ID":"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7","Type":"ContainerStarted","Data":"10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97"}
Oct 30 20:46:07.797522 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.797317    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" event={"ID":"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7","Type":"ContainerStarted","Data":"e4c8483e4bd3a7de4287da4593edf5f98b4934c230e51e0ac0d708159298f82b"}
Oct 30 20:46:07.821440 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:07.821362    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podStartSLOduration=1.821346111 podStartE2EDuration="1.821346111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:46:07.818739201 +0000 UTC m=+475.933473393" watchObservedRunningTime="2026-10-30 20:46:07.821346111 +0000 UTC m=+475.936080301"
Oct 30 20:46:08.321549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:08.321515    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:08.323037 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:08.323004    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 20:46:08.323185 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:08.323072    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 20:46:09.320984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:09.320944    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 20:46:09.321148 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:09.321003    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 20:46:10.063568 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:10.063532    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:46:10.083219 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:10.083194    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:46:10.321089 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:10.321004    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 20:46:10.321089 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:10.321070    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 20:46:10.823740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:10.823715    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-x27k5"
Oct 30 20:46:11.038255 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:11.038221    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:11.075163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:11.075063    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:11.321790 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:11.321756    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 20:46:11.321974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:11.321809    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 20:46:11.829097 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:11.829063    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd"
Oct 30 20:46:12.321851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:12.321814    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 20:46:12.322235 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:12.321870    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 20:46:13.321239 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:13.321201    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 20:46:13.321458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:13.321256    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 20:46:14.320974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:14.320940    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 20:46:14.321378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:14.320995    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 20:46:14.707407 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:14.707304    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:14.719998 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:14.719974    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:14.845944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:14.845911    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:46:15.757690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:15.757647    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:15.860510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:15.860472    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:15.937023 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:15.936988    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:15.937717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:15.937692    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:15.938569 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:15.938544    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 20:46:21.001069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:21.001026    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:21.117133 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:21.117098    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:21.556985 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:46:21.556946    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:46:22.495497 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:22.495464    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:22.595658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:22.595623    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:51.556455 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:46:51.556418    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:46:52.471722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:52.471687    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:53.035716 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.035680    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"]
Oct 30 20:46:53.039187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.039161    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.046897 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.046876    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-67598f1d-openshift-default-dockercfg-xt4jd\""
Oct 30 20:46:53.062599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.062572    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"]
Oct 30 20:46:53.172919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.172885    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-certs\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173047    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-envoy\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173091    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-socket\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173126    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-data\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173327 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173300    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfdjf\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-kube-api-access-vfdjf\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173354    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-token\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173381    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-credential-socket\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173920 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173521    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-podinfo\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.173920 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.173576    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5079682a-1b91-4bbf-b05c-d521f51e499b-istiod-ca-cert\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.274948 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.274914    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-envoy\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.274948 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.274954    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-socket\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.274979    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-data\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275004    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfdjf\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-kube-api-access-vfdjf\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275129    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-token\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275182    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-credential-socket\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275242    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-podinfo\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275291    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5079682a-1b91-4bbf-b05c-d521f51e499b-istiod-ca-cert\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275333    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-certs\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275431    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-data\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275666 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275476    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-socket\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275666 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275595    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-credential-socket\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.275771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.275702    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-certs\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.276287 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.276262    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5079682a-1b91-4bbf-b05c-d521f51e499b-istiod-ca-cert\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.277368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.277339    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-envoy\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.277818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.277800    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-podinfo\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.291523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.291454    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-token\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.292755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.292726    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfdjf\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-kube-api-access-vfdjf\") pod \"e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") " pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.352276 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.352240    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:53.664444 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.664333    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"]
Oct 30 20:46:53.668230 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:53.668184    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5079682a_1b91_4bbf_b05c_d521f51e499b.slice/crio-01513d61583e2c1d3e635839407b6765403e44c7536770222059f8bf1ac29c2c WatchSource:0}: Error finding container 01513d61583e2c1d3e635839407b6765403e44c7536770222059f8bf1ac29c2c: Status 404 returned error can't find the container with id 01513d61583e2c1d3e635839407b6765403e44c7536770222059f8bf1ac29c2c
Oct 30 20:46:53.670367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.670330    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:53.670474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.670457    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:53.670528 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.670521    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:53.783968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:53.783935    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:54.037674 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.037639    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" event={"ID":"5079682a-1b91-4bbf-b05c-d521f51e499b","Type":"ContainerStarted","Data":"9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392"}
Oct 30 20:46:54.038112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.037694    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" event={"ID":"5079682a-1b91-4bbf-b05c-d521f51e499b","Type":"ContainerStarted","Data":"01513d61583e2c1d3e635839407b6765403e44c7536770222059f8bf1ac29c2c"}
Oct 30 20:46:54.066289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.066232    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podStartSLOduration=1.066215873 podStartE2EDuration="1.066215873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:46:54.061127296 +0000 UTC m=+522.175861485" watchObservedRunningTime="2026-10-30 20:46:54.066215873 +0000 UTC m=+522.180950063"
Oct 30 20:46:54.351379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.351289    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:54.353274 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.353257    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:54.358161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.358137    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:54.729782 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.729694    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"]
Oct 30 20:46:54.734200 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.734174    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.741117 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.741093    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b16c21be-openshift-default-dockercfg-lxw4f\""
Oct 30 20:46:54.744658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.744632    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"]
Oct 30 20:46:54.780514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.780371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:54.787878 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.787849    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-data\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.787889    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-podinfo\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.787907    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-token\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.787982    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istiod-ca-cert\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.788044    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrc4f\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-kube-api-access-vrc4f\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.788091    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-envoy\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.788109    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-certs\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788241 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.788181    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-socket\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.788241 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.788199    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-credential-socket\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889039    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-podinfo\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889086    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-token\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889200    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istiod-ca-cert\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889310    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrc4f\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-kube-api-access-vrc4f\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889461 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889443    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-envoy\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889477    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-certs\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889647 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889621    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-socket\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889665    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-credential-socket\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889776 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889714    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-data\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.889934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889908    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-certs\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.890019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889969    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-socket\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.890019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.889973    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istiod-ca-cert\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.890107 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.890036    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-credential-socket\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.890142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.890099    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-data\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.891634 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.891608    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-envoy\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.891736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.891616    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-podinfo\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.897937 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.897912    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-token\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.898184 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.898169    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrc4f\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-kube-api-access-vrc4f\") pod \"e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") " pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:54.980048 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:54.979961    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:55.043926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.043893    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:55.045010 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.044992    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:46:55.053212 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.053191    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:55.354540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.354514    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"]
Oct 30 20:46:55.356084 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:55.356054    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd263061_7b8f_49b4_b1bf_9c1f7c579af7.slice/crio-175c82448772f6d251120b7f98e4782669364b5a0a30d37e4971ac412419f337 WatchSource:0}: Error finding container 175c82448772f6d251120b7f98e4782669364b5a0a30d37e4971ac412419f337: Status 404 returned error can't find the container with id 175c82448772f6d251120b7f98e4782669364b5a0a30d37e4971ac412419f337
Oct 30 20:46:55.358421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.358349    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:55.358540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.358434    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:55.358540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.358464    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:46:55.527261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.527232    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:55.842361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:55.842318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:56.040210 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.040177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:56.050238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.050203    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" event={"ID":"fd263061-7b8f-49b4-b1bf-9c1f7c579af7","Type":"ContainerStarted","Data":"019f4c6a3c5ae1581cfc55b4718035df880f0b6c9529d3abb98ae924610bb85a"}
Oct 30 20:46:56.050670 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.050258    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" event={"ID":"fd263061-7b8f-49b4-b1bf-9c1f7c579af7","Type":"ContainerStarted","Data":"175c82448772f6d251120b7f98e4782669364b5a0a30d37e4971ac412419f337"}
Oct 30 20:46:56.053790 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.053774    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:56.058855 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.058832    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:56.107224 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.107130    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podStartSLOduration=2.107116478 podStartE2EDuration="2.107116478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:46:56.103566864 +0000 UTC m=+524.218301078" watchObservedRunningTime="2026-10-30 20:46:56.107116478 +0000 UTC m=+524.221850668"
Oct 30 20:46:56.748072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.748041    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:56.858961 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:56.858925    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:57.055586 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.055554    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:57.056510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.056490    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:46:57.530929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.530877    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:57.663676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.663632    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:57.665599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.665576    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"]
Oct 30 20:46:57.668987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.668968    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:57.676253 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.676091    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-67598f1d-dockercfg-2wqdv\""
Oct 30 20:46:57.679909 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.679880    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"]
Oct 30 20:46:57.706924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.706893    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-67598f1d\""
Oct 30 20:46:57.716490 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.716444    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f10d8fa0-b207-49db-86ad-a523b020c31e-plugins-config-volume\") pod \"payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") " pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:57.716669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.716578    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb2bx\" (UniqueName: \"kubernetes.io/projected/f10d8fa0-b207-49db-86ad-a523b020c31e-kube-api-access-cb2bx\") pod \"payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") " pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:57.817527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.817440    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb2bx\" (UniqueName: \"kubernetes.io/projected/f10d8fa0-b207-49db-86ad-a523b020c31e-kube-api-access-cb2bx\") pod \"payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") " pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:57.817700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.817530    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f10d8fa0-b207-49db-86ad-a523b020c31e-plugins-config-volume\") pod \"payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") " pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:57.818674 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.818649    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f10d8fa0-b207-49db-86ad-a523b020c31e-plugins-config-volume\") pod \"payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") " pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:57.826584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.826556    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb2bx\" (UniqueName: \"kubernetes.io/projected/f10d8fa0-b207-49db-86ad-a523b020c31e-kube-api-access-cb2bx\") pod \"payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") " pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:57.933578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.933525    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:57.984361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:57.984323    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:46:58.039174 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.039050    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:58.145589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.145508    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:58.233556 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.233524    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:58.278419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.262615    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"]
Oct 30 20:46:58.583511 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.583478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:58.686351 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.686316    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"]
Oct 30 20:46:58.691226 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.691207    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:58.697334 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.697313    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-8f46c771\""
Oct 30 20:46:58.698267 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.697975    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-8f46c771-dockercfg-75wc6\""
Oct 30 20:46:58.705296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.705273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:58.710283 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.710257    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"]
Oct 30 20:46:58.726978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.726946    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhxpk\" (UniqueName: \"kubernetes.io/projected/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-kube-api-access-qhxpk\") pod \"payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:58.727142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.726992    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:58.758471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.758432    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"]
Oct 30 20:46:58.766409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.766335    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:58.779116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.777953    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"]
Oct 30 20:46:58.828576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.828539    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:58.828775 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.828582    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a27ab3c-e441-4100-8b5b-e9023c56762b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") " pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:58.828775 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.828659    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfvvk\" (UniqueName: \"kubernetes.io/projected/2a27ab3c-e441-4100-8b5b-e9023c56762b-kube-api-access-lfvvk\") pod \"payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") " pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:58.828860 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.828773    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhxpk\" (UniqueName: \"kubernetes.io/projected/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-kube-api-access-qhxpk\") pod \"payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:58.829375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.829355    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:58.841679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.841621    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhxpk\" (UniqueName: \"kubernetes.io/projected/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-kube-api-access-qhxpk\") pod \"payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:58.929766 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.929687    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a27ab3c-e441-4100-8b5b-e9023c56762b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") " pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:58.929943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.929771    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfvvk\" (UniqueName: \"kubernetes.io/projected/2a27ab3c-e441-4100-8b5b-e9023c56762b-kube-api-access-lfvvk\") pod \"payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") " pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:58.930590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.930566    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a27ab3c-e441-4100-8b5b-e9023c56762b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") " pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:58.941450 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:58.941420    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfvvk\" (UniqueName: \"kubernetes.io/projected/2a27ab3c-e441-4100-8b5b-e9023c56762b-kube-api-access-lfvvk\") pod \"payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") " pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:59.005484 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.005451    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:46:59.077427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.076866    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv" event={"ID":"f10d8fa0-b207-49db-86ad-a523b020c31e","Type":"ContainerStarted","Data":"41df407a1b04d31e18f085e2cc9b1984c3654c09f98bf35434987a59d2215735"}
Oct 30 20:46:59.095192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.095120    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:46:59.238054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.238018    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:59.309319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.309270    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"]
Oct 30 20:46:59.311920 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:59.311887    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad00ec3_72b1_4f47_9207_646bf9fdd50c.slice/crio-25cf6874f3de4b107f37653b0646242ff031ec661ee3762e9519047a79dba610 WatchSource:0}: Error finding container 25cf6874f3de4b107f37653b0646242ff031ec661ee3762e9519047a79dba610: Status 404 returned error can't find the container with id 25cf6874f3de4b107f37653b0646242ff031ec661ee3762e9519047a79dba610
Oct 30 20:46:59.399523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.399489    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"]
Oct 30 20:46:59.401758 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:46:59.401719    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a27ab3c_e441_4100_8b5b_e9023c56762b.slice/crio-8f8149c5e15b02e74c2a9e6322606d13e8959cba3753e2589b681760d10e8511 WatchSource:0}: Error finding container 8f8149c5e15b02e74c2a9e6322606d13e8959cba3753e2589b681760d10e8511: Status 404 returned error can't find the container with id 8f8149c5e15b02e74c2a9e6322606d13e8959cba3753e2589b681760d10e8511
Oct 30 20:46:59.576814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.576777    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:59.607992 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.607961    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"]
Oct 30 20:46:59.612868 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.612844    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:46:59.621965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.621938    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"]
Oct 30 20:46:59.623262 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.623234    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b16c21be\""
Oct 30 20:46:59.638085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.638053    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmxw\" (UniqueName: \"kubernetes.io/projected/83015a0c-51f7-4382-8b4d-2b5d24f95b82-kube-api-access-9zmxw\") pod \"payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:46:59.638244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.638124    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83015a0c-51f7-4382-8b4d-2b5d24f95b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:46:59.676581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.676497    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:46:59.743154 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.741135    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmxw\" (UniqueName: \"kubernetes.io/projected/83015a0c-51f7-4382-8b4d-2b5d24f95b82-kube-api-access-9zmxw\") pod \"payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:46:59.743154 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.741272    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83015a0c-51f7-4382-8b4d-2b5d24f95b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:46:59.743154 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.742587    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83015a0c-51f7-4382-8b4d-2b5d24f95b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:46:59.765423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.763721    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmxw\" (UniqueName: \"kubernetes.io/projected/83015a0c-51f7-4382-8b4d-2b5d24f95b82-kube-api-access-9zmxw\") pod \"payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:46:59.930626 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:46:59.930547    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:47:00.085971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:00.085934    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc" event={"ID":"2a27ab3c-e441-4100-8b5b-e9023c56762b","Type":"ContainerStarted","Data":"8f8149c5e15b02e74c2a9e6322606d13e8959cba3753e2589b681760d10e8511"}
Oct 30 20:47:00.087645 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:00.087612    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62" event={"ID":"2ad00ec3-72b1-4f47-9207-646bf9fdd50c","Type":"ContainerStarted","Data":"25cf6874f3de4b107f37653b0646242ff031ec661ee3762e9519047a79dba610"}
Oct 30 20:47:00.225042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:00.224185    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"]
Oct 30 20:47:00.241580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:00.241535    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:00.336997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:00.336944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:00.437661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:00.437627    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:01.097756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:01.097718    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n" event={"ID":"83015a0c-51f7-4382-8b4d-2b5d24f95b82","Type":"ContainerStarted","Data":"6a5c44aa8f153c1067e3c89d7a1f9f1028e54723a4dd0444feae826f494b4ac3"}
Oct 30 20:47:01.252924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:01.252885    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:01.363210 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:01.359092    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:01.801757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:01.801726    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:01.889514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:01.889480    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:02.105105 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.105068    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc" event={"ID":"2a27ab3c-e441-4100-8b5b-e9023c56762b","Type":"ContainerStarted","Data":"a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3"}
Oct 30 20:47:02.105326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.105154    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:47:02.106658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.106631    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n" event={"ID":"83015a0c-51f7-4382-8b4d-2b5d24f95b82","Type":"ContainerStarted","Data":"b24eb6f0216b7f885856e0324d47875f01a58deead151765da15a7816fc7bc37"}
Oct 30 20:47:02.106784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.106688    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:47:02.108071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.108046    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62" event={"ID":"2ad00ec3-72b1-4f47-9207-646bf9fdd50c","Type":"ContainerStarted","Data":"59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0"}
Oct 30 20:47:02.108218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.108088    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:47:02.109862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.109835    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv" event={"ID":"f10d8fa0-b207-49db-86ad-a523b020c31e","Type":"ContainerStarted","Data":"675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895"}
Oct 30 20:47:02.109986 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.109968    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:47:02.123348 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.123295    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc" podStartSLOduration=2.200147393 podStartE2EDuration="4.123277368s" totalImagesPullingTime="1.923129975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:02.122957136 +0000 UTC m=+530.237691327" watchObservedRunningTime="2026-10-30 20:47:02.123277368 +0000 UTC m=+530.238011559"
Oct 30 20:47:02.123542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.123444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:02.139372 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.139176    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n" podStartSLOduration=1.646957539 podStartE2EDuration="3.139160384s" totalImagesPullingTime="1.492202845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:02.139036226 +0000 UTC m=+530.253770416" watchObservedRunningTime="2026-10-30 20:47:02.139160384 +0000 UTC m=+530.253894571"
Oct 30 20:47:02.159958 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.159892    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62" podStartSLOduration=2.100231552 podStartE2EDuration="4.159873514s" totalImagesPullingTime="2.059641962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:02.156638899 +0000 UTC m=+530.271373103" watchObservedRunningTime="2026-10-30 20:47:02.159873514 +0000 UTC m=+530.274607706"
Oct 30 20:47:02.180707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.180648    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv" podStartSLOduration=2.200565676 podStartE2EDuration="5.180634886s" totalImagesPullingTime="2.98006921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:02.178440614 +0000 UTC m=+530.293174805" watchObservedRunningTime="2026-10-30 20:47:02.180634886 +0000 UTC m=+530.295369093"
Oct 30 20:47:02.220406 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.220363    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:02.548263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.548228    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"]
Oct 30 20:47:02.552013 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.551987    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.571505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.571481    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-eaf151e7-openshift-default-dockercfg-6jfg2\""
Oct 30 20:47:02.578589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578565    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vfz5\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-kube-api-access-8vfz5\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.578735 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578596    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-certs\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.578735 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578623    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-envoy\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.578735 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578640    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-credential-socket\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.578897 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578806    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-data\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.578897 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578856    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istiod-ca-cert\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.579001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578923    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-socket\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.579001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.578969    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-podinfo\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.579101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.579056    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-token\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.587784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.587753    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"]
Oct 30 20:47:02.680530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680489    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vfz5\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-kube-api-access-8vfz5\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.680530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680532    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-certs\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.680788 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680572    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-envoy\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.680788 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680603    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-credential-socket\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.680788 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680659    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-data\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.680788 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680692    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istiod-ca-cert\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.680788 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680720    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-socket\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.680788 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680751    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-podinfo\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.681088 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.680806    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-token\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.681088 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.681048    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-certs\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.681336 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.681313    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-socket\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.681602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.681585    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-credential-socket\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.681676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.681617    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istiod-ca-cert\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.681841 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.681824    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-data\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.683330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.683310    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-envoy\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.683687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.683667    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-podinfo\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.693507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.693472    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-token\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.694044 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.694011    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vfz5\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-kube-api-access-8vfz5\") pod \"e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") " pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.756795 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.756733    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:02.864500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.864419    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:02.868083 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:02.868051    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:03.065741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.065699    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:03.147644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.147620    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"]
Oct 30 20:47:03.149028 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:03.149002    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20cf0a5b_6b0c_4b84_b772_80e6b7edaf7b.slice/crio-371c0259308fd356136aa6afc08d686635cc440065bb87ac644a3d5100b06177 WatchSource:0}: Error finding container 371c0259308fd356136aa6afc08d686635cc440065bb87ac644a3d5100b06177: Status 404 returned error can't find the container with id 371c0259308fd356136aa6afc08d686635cc440065bb87ac644a3d5100b06177
Oct 30 20:47:03.151298 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.151263    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:03.151381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.151348    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:03.151456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.151379    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:03.165612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.165592    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:03.202766 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.202741    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"]
Oct 30 20:47:03.214805 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.214783    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.215772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.215723    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"]
Oct 30 20:47:03.219972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.219778    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b16c21be-serving-cert\""
Oct 30 20:47:03.219972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.219868    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 20:47:03.219972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.219909    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 20:47:03.219972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.219932    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbd9l\""
Oct 30 20:47:03.220235 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.220034    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 20:47:03.263772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.263721    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:03.287125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.287094    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-openshift-service-ca\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.287301 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.287137    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-maas-api-tls\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.287301 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.287243    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s55jh\" (UniqueName: \"kubernetes.io/projected/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-kube-api-access-s55jh\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.287419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.287376    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-metrics-certs\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.365872 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.365839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:03.388065 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.388035    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-openshift-service-ca\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.388261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.388075    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-maas-api-tls\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.388261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.388098    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s55jh\" (UniqueName: \"kubernetes.io/projected/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-kube-api-access-s55jh\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.388261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.388150    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-metrics-certs\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.388877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.388847    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-openshift-service-ca\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.390630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.390609    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-maas-api-tls\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.390749 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.390643    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-metrics-certs\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.397617 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.397566    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s55jh\" (UniqueName: \"kubernetes.io/projected/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-kube-api-access-s55jh\") pod \"maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.465033 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.465002    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:03.532290 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.532254    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:03.809789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:03.809759    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"]
Oct 30 20:47:03.811182 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:03.811153    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fcd9f66_2232_49df_a3fd_e1be5ed69a82.slice/crio-4464129619e8eea65f5be3a4f49af585e0f6bda9dbfc7960f3e85373dc4daddb WatchSource:0}: Error finding container 4464129619e8eea65f5be3a4f49af585e0f6bda9dbfc7960f3e85373dc4daddb: Status 404 returned error can't find the container with id 4464129619e8eea65f5be3a4f49af585e0f6bda9dbfc7960f3e85373dc4daddb
Oct 30 20:47:04.122499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:04.122412    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx" event={"ID":"3fcd9f66-2232-49df-a3fd-e1be5ed69a82","Type":"ContainerStarted","Data":"4464129619e8eea65f5be3a4f49af585e0f6bda9dbfc7960f3e85373dc4daddb"}
Oct 30 20:47:04.124216 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:04.124185    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" event={"ID":"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b","Type":"ContainerStarted","Data":"96c7dfe133eeaf37813d4874e0e6da94abd1bdb196eff07392f06b73de1ea0ec"}
Oct 30 20:47:04.124337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:04.124241    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" event={"ID":"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b","Type":"ContainerStarted","Data":"371c0259308fd356136aa6afc08d686635cc440065bb87ac644a3d5100b06177"}
Oct 30 20:47:04.155131 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:04.155070    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" podStartSLOduration=2.155053729 podStartE2EDuration="2.155053729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:04.154362136 +0000 UTC m=+532.269096328" watchObservedRunningTime="2026-10-30 20:47:04.155053729 +0000 UTC m=+532.269787920"
Oct 30 20:47:04.401584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:04.401502    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:04.874562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:04.874519    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:04.881691 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:04.881660    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:05.131215 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:05.131139    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:05.132134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:05.132111    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:05.737119 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:05.737077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:05.835194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:05.835164    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:07.111689 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.111589    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:07.147170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.147136    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx" event={"ID":"3fcd9f66-2232-49df-a3fd-e1be5ed69a82","Type":"ContainerStarted","Data":"fb777edc9ba6c1867e0707f3c98bd5de0a739839211eafe500d7f8e29c5f5c87"}
Oct 30 20:47:07.147333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.147296    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:07.180439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.180367    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx" podStartSLOduration=1.156261919 podStartE2EDuration="4.180353419s" totalImagesPullingTime="3.0240915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:07.179661769 +0000 UTC m=+535.294395982" watchObservedRunningTime="2026-10-30 20:47:07.180353419 +0000 UTC m=+535.295087608"
Oct 30 20:47:07.192923 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.192892    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"]
Oct 30 20:47:07.196500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.196474    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.204958 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.204936    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-eaf151e7-dockercfg-r56bd\""
Oct 30 20:47:07.205069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.205052    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-eaf151e7\""
Oct 30 20:47:07.210132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.210110    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"]
Oct 30 20:47:07.215678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.215637    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:07.221123 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.216828    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"]
Oct 30 20:47:07.226134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.226106    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.229497 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.229468    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ebbd3a-8954-455d-96ec-c159377f315b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.230020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.229525    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvffm\" (UniqueName: \"kubernetes.io/projected/35ebbd3a-8954-455d-96ec-c159377f315b-kube-api-access-lvffm\") pod \"payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.235354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.235328    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"]
Oct 30 20:47:07.334428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.330210    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvffm\" (UniqueName: \"kubernetes.io/projected/35ebbd3a-8954-455d-96ec-c159377f315b-kube-api-access-lvffm\") pod \"payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.334428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.330399    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28437a12-dc09-467d-ac94-34172643e99e-plugins-config-volume\") pod \"payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") " pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.334428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.330503    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ebbd3a-8954-455d-96ec-c159377f315b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.334428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.330552    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfv84\" (UniqueName: \"kubernetes.io/projected/28437a12-dc09-467d-ac94-34172643e99e-kube-api-access-sfv84\") pod \"payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") " pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.334428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.334186    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ebbd3a-8954-455d-96ec-c159377f315b-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.340118 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.340061    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvffm\" (UniqueName: \"kubernetes.io/projected/35ebbd3a-8954-455d-96ec-c159377f315b-kube-api-access-lvffm\") pod \"payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.431744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.431642    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28437a12-dc09-467d-ac94-34172643e99e-plugins-config-volume\") pod \"payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") " pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.431744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.431714    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfv84\" (UniqueName: \"kubernetes.io/projected/28437a12-dc09-467d-ac94-34172643e99e-kube-api-access-sfv84\") pod \"payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") " pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.432375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.432354    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28437a12-dc09-467d-ac94-34172643e99e-plugins-config-volume\") pod \"payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") " pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.440516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.440492    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfv84\" (UniqueName: \"kubernetes.io/projected/28437a12-dc09-467d-ac94-34172643e99e-kube-api-access-sfv84\") pod \"payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") " pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.512616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.511295    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:07.545202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.545168    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:07.845975 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.845944    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"]
Oct 30 20:47:07.847705 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:07.847681    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ebbd3a_8954_455d_96ec_c159377f315b.slice/crio-0a84d6e38fe7c3323e21b94582e7959bc7d80400dbb525cfe6f9d4e2e0a3f0c0 WatchSource:0}: Error finding container 0a84d6e38fe7c3323e21b94582e7959bc7d80400dbb525cfe6f9d4e2e0a3f0c0: Status 404 returned error can't find the container with id 0a84d6e38fe7c3323e21b94582e7959bc7d80400dbb525cfe6f9d4e2e0a3f0c0
Oct 30 20:47:07.917575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:07.917546    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"]
Oct 30 20:47:07.920700 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:07.920672    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28437a12_dc09_467d_ac94_34172643e99e.slice/crio-b8783ddbe6a9d583beefcee362a567f6741ce76264f6569364f82179a8ecfe08 WatchSource:0}: Error finding container b8783ddbe6a9d583beefcee362a567f6741ce76264f6569364f82179a8ecfe08: Status 404 returned error can't find the container with id b8783ddbe6a9d583beefcee362a567f6741ce76264f6569364f82179a8ecfe08
Oct 30 20:47:08.201467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.201352    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx" event={"ID":"28437a12-dc09-467d-ac94-34172643e99e","Type":"ContainerStarted","Data":"9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341"}
Oct 30 20:47:08.201957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.201451    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx" event={"ID":"28437a12-dc09-467d-ac94-34172643e99e","Type":"ContainerStarted","Data":"b8783ddbe6a9d583beefcee362a567f6741ce76264f6569364f82179a8ecfe08"}
Oct 30 20:47:08.202369 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.202347    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:08.205097 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.205074    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d" event={"ID":"35ebbd3a-8954-455d-96ec-c159377f315b","Type":"ContainerStarted","Data":"f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64"}
Oct 30 20:47:08.205219 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.205117    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d" event={"ID":"35ebbd3a-8954-455d-96ec-c159377f315b","Type":"ContainerStarted","Data":"0a84d6e38fe7c3323e21b94582e7959bc7d80400dbb525cfe6f9d4e2e0a3f0c0"}
Oct 30 20:47:08.205304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.205283    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:08.219729 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.219679    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx" podStartSLOduration=1.219663617 podStartE2EDuration="1.219663617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:08.218503074 +0000 UTC m=+536.333237265" watchObservedRunningTime="2026-10-30 20:47:08.219663617 +0000 UTC m=+536.334397807"
Oct 30 20:47:08.236681 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.236629    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d" podStartSLOduration=1.236613107 podStartE2EDuration="1.236613107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:08.23389925 +0000 UTC m=+536.348633442" watchObservedRunningTime="2026-10-30 20:47:08.236613107 +0000 UTC m=+536.351347296"
Oct 30 20:47:08.278411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:08.274944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:09.617200 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:09.617170    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:10.569108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:10.569007    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:11.369408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:11.364989    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:11.444813 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:11.444765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:11.603967 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:11.603923    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:12.609198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:12.609155    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:13.118410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:13.118052    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:47:13.118600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:13.118500    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:47:13.118600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:13.118557    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:47:13.118600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:13.118594    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:47:13.214147 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:13.214117    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:13.837974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:13.837938    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:14.275662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:14.275622    4852 generic.go:356] "Generic (PLEG): container finished" podID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerID="9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392" exitCode=0
Oct 30 20:47:14.275922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:14.275832    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" event={"ID":"5079682a-1b91-4bbf-b05c-d521f51e499b","Type":"ContainerDied","Data":"9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392"}
Oct 30 20:47:14.276267 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:14.276243    4852 scope.go:122] "RemoveContainer" containerID="9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392"
Oct 30 20:47:14.277641 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:14.277611    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:14.277745 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:14.277677    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:14.277745 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:14.277707    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:14.942189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:14.942158    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:15.310186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:15.309422    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" event={"ID":"5079682a-1b91-4bbf-b05c-d521f51e499b","Type":"ContainerStarted","Data":"f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1"}
Oct 30 20:47:15.374277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:15.373817    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:47:15.374277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:15.373944    4852 patch_prober.go:29] interesting pod/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 20:47:15.374277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:15.374002    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 20:47:16.075411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.075290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:16.279410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.278930    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:16.372482 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.372361    4852 patch_prober.go:29] interesting pod/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 20:47:16.372482 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.372443    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 20:47:16.383071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.382979    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:16.414924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.414888    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"]
Oct 30 20:47:16.418521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.418505    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.424932 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.424903    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:47:16.425122 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.425101    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:47:16.427579 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.427554    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"]
Oct 30 20:47:16.512268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.512222    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwdmg\" (UniqueName: \"kubernetes.io/projected/c47dbd4b-61c4-4944-819b-13d260bb0c28-kube-api-access-qwdmg\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.512268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.512270    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.512532 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.512452    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.613204 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.613159    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.613441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.613293    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwdmg\" (UniqueName: \"kubernetes.io/projected/c47dbd4b-61c4-4944-819b-13d260bb0c28-kube-api-access-qwdmg\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.613441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.613328    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.613869 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.613843    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.616039 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.616008    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.645058 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.644981    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwdmg\" (UniqueName: \"kubernetes.io/projected/c47dbd4b-61c4-4944-819b-13d260bb0c28-kube-api-access-qwdmg\") pod \"maas-api-revoke-keys-e2e-ait-67598f1d-4k28j\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:16.736695 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:16.736653    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:17.110227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:17.110195    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"]
Oct 30 20:47:17.112637 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:17.112583    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc47dbd4b_61c4_4944_819b_13d260bb0c28.slice/crio-2205e25179440e06f4a223a279fa41467755354f0f60022e395b54367bc5fb90 WatchSource:0}: Error finding container 2205e25179440e06f4a223a279fa41467755354f0f60022e395b54367bc5fb90: Status 404 returned error can't find the container with id 2205e25179440e06f4a223a279fa41467755354f0f60022e395b54367bc5fb90
Oct 30 20:47:17.322014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:17.321982    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j" event={"ID":"c47dbd4b-61c4-4944-819b-13d260bb0c28","Type":"ContainerStarted","Data":"2205e25179440e06f4a223a279fa41467755354f0f60022e395b54367bc5fb90"}
Oct 30 20:47:17.325083 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:17.325038    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:17.366549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:17.354336    4852 patch_prober.go:29] interesting pod/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 20:47:17.366549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:17.354427    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 20:47:18.296424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.295728    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:18.329366 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.329332    4852 generic.go:356] "Generic (PLEG): container finished" podID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerID="019f4c6a3c5ae1581cfc55b4718035df880f0b6c9529d3abb98ae924610bb85a" exitCode=0
Oct 30 20:47:18.329586 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.329431    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" event={"ID":"fd263061-7b8f-49b4-b1bf-9c1f7c579af7","Type":"ContainerDied","Data":"019f4c6a3c5ae1581cfc55b4718035df880f0b6c9529d3abb98ae924610bb85a"}
Oct 30 20:47:18.329970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.329891    4852 scope.go:122] "RemoveContainer" containerID="019f4c6a3c5ae1581cfc55b4718035df880f0b6c9529d3abb98ae924610bb85a"
Oct 30 20:47:18.331789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.331760    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:18.331953 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.331938    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:18.332040 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.332030    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:18.417698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.417666    4852 patch_prober.go:29] interesting pod/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 20:47:18.418019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.417993    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 20:47:18.436813 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:18.436782    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:19.104062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:19.103954    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:19.229475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:19.229443    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:19.343216 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:19.343175    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" event={"ID":"fd263061-7b8f-49b4-b1bf-9c1f7c579af7","Type":"ContainerStarted","Data":"0743524eb2adcb8e9ebb5815cd64f3cf2df98fe615b9a268fcea4c98da1a3f5d"}
Oct 30 20:47:19.360468 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:19.357589    4852 patch_prober.go:29] interesting pod/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 20:47:19.360468 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:19.357664    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 20:47:20.033406 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.032642    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:20.064132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.063432    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:47:20.088425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.085999    4852 patch_prober.go:29] interesting pod/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 20:47:20.088425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.086063    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 20:47:20.236413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.235531    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:20.370530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.364617    4852 patch_prober.go:29] interesting pod/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 30 20:47:20.370530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.364674    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 30 20:47:20.740475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:20.738258    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:21.071411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.069490    4852 patch_prober.go:29] interesting pod/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 20:47:21.071411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.069551    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 20:47:21.379340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.379265    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:47:21.379830 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.379809    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:47:21.381057 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.380725    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:47:21.545852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.545821    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:21.594465 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:21.593295    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:47:21.828658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.828624    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:21.927924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:21.927884    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:22.063921 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:22.062771    4852 patch_prober.go:29] interesting pod/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 20:47:22.063921 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:22.062833    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 20:47:22.433579 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:22.433543    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:23.057319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:23.056832    4852 patch_prober.go:29] interesting pod/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 20:47:23.057319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:23.056898    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 20:47:23.220422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:23.220377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:23.478600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:23.478555    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j" event={"ID":"c47dbd4b-61c4-4944-819b-13d260bb0c28","Type":"ContainerStarted","Data":"295b69f13ee78dfe8f5f614de9d429d06ad6f87334f6fa13acb026607a40b36b"}
Oct 30 20:47:23.499536 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:23.499476    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j" podStartSLOduration=2.296877335 podStartE2EDuration="7.499462554s" totalImagesPullingTime="5.202585219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:23.49733897 +0000 UTC m=+551.612073161" watchObservedRunningTime="2026-10-30 20:47:23.499462554 +0000 UTC m=+551.614196743"
Oct 30 20:47:24.069112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:24.069077    4852 patch_prober.go:29] interesting pod/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 30 20:47:24.069439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:24.069376    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 30 20:47:24.135939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:24.135907    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:24.487294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:24.487208    4852 generic.go:356] "Generic (PLEG): container finished" podID="c47dbd4b-61c4-4944-819b-13d260bb0c28" containerID="295b69f13ee78dfe8f5f614de9d429d06ad6f87334f6fa13acb026607a40b36b" exitCode=0
Oct 30 20:47:24.487294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:24.487276    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j" event={"ID":"c47dbd4b-61c4-4944-819b-13d260bb0c28","Type":"ContainerDied","Data":"295b69f13ee78dfe8f5f614de9d429d06ad6f87334f6fa13acb026607a40b36b"}
Oct 30 20:47:24.939834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:24.939801    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:25.058082 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.058054    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:47:25.058408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.058375    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:47:25.059337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.059318    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:47:25.494306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.494224    4852 generic.go:356] "Generic (PLEG): container finished" podID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerID="96c7dfe133eeaf37813d4874e0e6da94abd1bdb196eff07392f06b73de1ea0ec" exitCode=0
Oct 30 20:47:25.494839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.494301    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" event={"ID":"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b","Type":"ContainerDied","Data":"96c7dfe133eeaf37813d4874e0e6da94abd1bdb196eff07392f06b73de1ea0ec"}
Oct 30 20:47:25.494839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.494831    4852 scope.go:122] "RemoveContainer" containerID="96c7dfe133eeaf37813d4874e0e6da94abd1bdb196eff07392f06b73de1ea0ec"
Oct 30 20:47:25.496120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.496090    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:25.496190 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.496162    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:25.496230 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.496203    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:25.650718 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.650690    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:25.723287 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.723252    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-xks-ca\") pod \"c47dbd4b-61c4-4944-819b-13d260bb0c28\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") "
Oct 30 20:47:25.723508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.723303    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c47dbd4b-61c4-4944-819b-13d260bb0c28-kube-api-access-qwdmg\" (UniqueName: \"kubernetes.io/projected/c47dbd4b-61c4-4944-819b-13d260bb0c28-kube-api-access-qwdmg\") pod \"c47dbd4b-61c4-4944-819b-13d260bb0c28\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") "
Oct 30 20:47:25.723508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.723352    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-openshift-ca\") pod \"c47dbd4b-61c4-4944-819b-13d260bb0c28\" (UID: \"c47dbd4b-61c4-4944-819b-13d260bb0c28\") "
Oct 30 20:47:25.724117 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.724067    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-openshift-ca" pod "c47dbd4b-61c4-4944-819b-13d260bb0c28" (UID: "c47dbd4b-61c4-4944-819b-13d260bb0c28"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:25.726042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.726010    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c47dbd4b-61c4-4944-819b-13d260bb0c28-kube-api-access-qwdmg" pod "c47dbd4b-61c4-4944-819b-13d260bb0c28" (UID: "c47dbd4b-61c4-4944-819b-13d260bb0c28"). InnerVolumeSpecName "kube-api-access-qwdmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:25.726293 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.726267    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-xks-ca" pod "c47dbd4b-61c4-4944-819b-13d260bb0c28" (UID: "c47dbd4b-61c4-4944-819b-13d260bb0c28"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:47:25.829864 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.829114    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:25.829864 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.829152    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwdmg\" (UniqueName: \"kubernetes.io/projected/c47dbd4b-61c4-4944-819b-13d260bb0c28-kube-api-access-qwdmg\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:25.829864 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.829169    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c47dbd4b-61c4-4944-819b-13d260bb0c28-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:25.861723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:25.861684    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:26.189992 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.189901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:26.282725 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.282690    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:26.380332 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.380296    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:26.502162 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.502077    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" event={"ID":"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b","Type":"ContainerStarted","Data":"80469cc272f51a62318a80a1f22f7965b1ae2c05fd28def08cde1ae8fdc8b687"}
Oct 30 20:47:26.503632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.503603    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j" event={"ID":"c47dbd4b-61c4-4944-819b-13d260bb0c28","Type":"ContainerDied","Data":"2205e25179440e06f4a223a279fa41467755354f0f60022e395b54367bc5fb90"}
Oct 30 20:47:26.503632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.503627    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"
Oct 30 20:47:26.503819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.503686    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2205e25179440e06f4a223a279fa41467755354f0f60022e395b54367bc5fb90"
Oct 30 20:47:26.867412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.865849    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:26.867412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.865980    4852 patch_prober.go:29] interesting pod/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:47:26.867412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:26.866022    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 20:47:27.090129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.090072    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"]
Oct 30 20:47:27.104120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.104088    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"]
Oct 30 20:47:27.139680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.139586    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:27.238566 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.238531    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:27.818136 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.818102    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:27.872413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.869846    4852 patch_prober.go:29] interesting pod/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:47:27.872413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.869907    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 20:47:27.981332 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:27.981295    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:28.671369 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:28.671320    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:28.837647 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:28.837611    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:28.870881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:28.870841    4852 patch_prober.go:29] interesting pod/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:47:28.871175 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:28.871144    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 20:47:29.146505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:29.146463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:29.286521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:29.286480    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:29.640375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:29.640328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:29.874798 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:29.874759    4852 patch_prober.go:29] interesting pod/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:47:29.875259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:29.874819    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 20:47:29.885671 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:29.885645    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:29.980885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:29.980800    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:30.090395 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:30.090280    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:30.183795 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:30.183756    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:30.865936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:30.865902    4852 patch_prober.go:29] interesting pod/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 20:47:30.866226 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:30.866197    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 20:47:30.881265 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:30.881232    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:30.982202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:30.982173    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:31.391287 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:31.391254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:31.649583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:31.649488    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:31.869596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:31.869568    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:31.869869 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:31.869849    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:31.870435 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:31.870418    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:32.046421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.046373    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:32.091193 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.091149    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc" podUID="2a27ab3c-e441-4100-8b5b-e9023c56762b" containerName="payload-processing" containerID="cri-o://a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3" gracePeriod=25
Oct 30 20:47:32.104723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.104692    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62" podUID="2ad00ec3-72b1-4f47-9207-646bf9fdd50c" containerName="payload-pre-processing" containerID="cri-o://59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0" gracePeriod=25
Oct 30 20:47:32.352312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.352281    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:47:32.389016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.388990    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:47:32.424655 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.424621    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a27ab3c-e441-4100-8b5b-e9023c56762b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a27ab3c-e441-4100-8b5b-e9023c56762b-plugins-config-volume\") pod \"2a27ab3c-e441-4100-8b5b-e9023c56762b\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") "
Oct 30 20:47:32.424816 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.424799    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-plugins-config-volume\") pod \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") "
Oct 30 20:47:32.424872 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.424857    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a27ab3c-e441-4100-8b5b-e9023c56762b-kube-api-access-lfvvk\" (UniqueName: \"kubernetes.io/projected/2a27ab3c-e441-4100-8b5b-e9023c56762b-kube-api-access-lfvvk\") pod \"2a27ab3c-e441-4100-8b5b-e9023c56762b\" (UID: \"2a27ab3c-e441-4100-8b5b-e9023c56762b\") "
Oct 30 20:47:32.424922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.424888    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-kube-api-access-qhxpk\" (UniqueName: \"kubernetes.io/projected/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-kube-api-access-qhxpk\") pod \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\" (UID: \"2ad00ec3-72b1-4f47-9207-646bf9fdd50c\") "
Oct 30 20:47:32.425195 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.425169    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a27ab3c-e441-4100-8b5b-e9023c56762b-plugins-config-volume" pod "2a27ab3c-e441-4100-8b5b-e9023c56762b" (UID: "2a27ab3c-e441-4100-8b5b-e9023c56762b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:32.425305 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.425205    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-plugins-config-volume" pod "2ad00ec3-72b1-4f47-9207-646bf9fdd50c" (UID: "2ad00ec3-72b1-4f47-9207-646bf9fdd50c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:32.425305 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.425298    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:32.425413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.425313    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a27ab3c-e441-4100-8b5b-e9023c56762b-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:32.427227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.427188    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a27ab3c-e441-4100-8b5b-e9023c56762b-kube-api-access-lfvvk" pod "2a27ab3c-e441-4100-8b5b-e9023c56762b" (UID: "2a27ab3c-e441-4100-8b5b-e9023c56762b"). InnerVolumeSpecName "kube-api-access-lfvvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:32.427227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.427213    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-kube-api-access-qhxpk" pod "2ad00ec3-72b1-4f47-9207-646bf9fdd50c" (UID: "2ad00ec3-72b1-4f47-9207-646bf9fdd50c"). InnerVolumeSpecName "kube-api-access-qhxpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:32.526199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.526162    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfvvk\" (UniqueName: \"kubernetes.io/projected/2a27ab3c-e441-4100-8b5b-e9023c56762b-kube-api-access-lfvvk\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:32.526199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.526200    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhxpk\" (UniqueName: \"kubernetes.io/projected/2ad00ec3-72b1-4f47-9207-646bf9fdd50c-kube-api-access-qhxpk\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:32.575279 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.575247    4852 generic.go:356] "Generic (PLEG): container finished" podID="2a27ab3c-e441-4100-8b5b-e9023c56762b" containerID="a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3" exitCode=0
Oct 30 20:47:32.575507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.575298    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"
Oct 30 20:47:32.575507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.575330    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc" event={"ID":"2a27ab3c-e441-4100-8b5b-e9023c56762b","Type":"ContainerDied","Data":"a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3"}
Oct 30 20:47:32.575507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.575380    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc" event={"ID":"2a27ab3c-e441-4100-8b5b-e9023c56762b","Type":"ContainerDied","Data":"8f8149c5e15b02e74c2a9e6322606d13e8959cba3753e2589b681760d10e8511"}
Oct 30 20:47:32.575507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.575440    4852 scope.go:122] "RemoveContainer" containerID="a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3"
Oct 30 20:47:32.576728 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.576707    4852 generic.go:356] "Generic (PLEG): container finished" podID="2ad00ec3-72b1-4f47-9207-646bf9fdd50c" containerID="59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0" exitCode=0
Oct 30 20:47:32.576801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.576749    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"
Oct 30 20:47:32.576801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.576763    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62" event={"ID":"2ad00ec3-72b1-4f47-9207-646bf9fdd50c","Type":"ContainerDied","Data":"59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0"}
Oct 30 20:47:32.576881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.576815    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62" event={"ID":"2ad00ec3-72b1-4f47-9207-646bf9fdd50c","Type":"ContainerDied","Data":"25cf6874f3de4b107f37653b0646242ff031ec661ee3762e9519047a79dba610"}
Oct 30 20:47:32.585125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.585104    4852 scope.go:122] "RemoveContainer" containerID="a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3"
Oct 30 20:47:32.585464 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:32.585444    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3\": container with ID starting with a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3 not found: ID does not exist" containerID="a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3"
Oct 30 20:47:32.585538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.585472    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3"} err="failed to get container status \"a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3\": rpc error: code = NotFound desc = could not find container \"a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3\": container with ID starting with a8777a0a021ce7a655f1399389943706bd35a5881a2b8287db23efd5511499a3 not found: ID does not exist"
Oct 30 20:47:32.585538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.585487    4852 scope.go:122] "RemoveContainer" containerID="59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0"
Oct 30 20:47:32.594801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.594783    4852 scope.go:122] "RemoveContainer" containerID="59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0"
Oct 30 20:47:32.595084 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:32.595065    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0\": container with ID starting with 59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0 not found: ID does not exist" containerID="59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0"
Oct 30 20:47:32.595132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.595093    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0"} err="failed to get container status \"59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0\": rpc error: code = NotFound desc = could not find container \"59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0\": container with ID starting with 59c97f7900727a3dcc1a62a081cfdb3cb4a533f37e66d4cbcf2f94b056c68ae0 not found: ID does not exist"
Oct 30 20:47:32.603111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.603089    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"]
Oct 30 20:47:32.605517 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.605496    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8f46c771-667cf6645c-hcqvc"]
Oct 30 20:47:32.608625 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.608607    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:32.615333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.615311    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"]
Oct 30 20:47:32.617808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.617787    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8f46c771-bbf76899f-52t62"]
Oct 30 20:47:32.713530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.713494    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:32.828150 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.828120    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:32.962375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:32.962272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:33.904107 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.904080    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:33.988886 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.988852    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"]
Oct 30 20:47:33.989311 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989298    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c47dbd4b-61c4-4944-819b-13d260bb0c28" containerName="revoke-keys"
Oct 30 20:47:33.989356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989313    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c47dbd4b-61c4-4944-819b-13d260bb0c28" containerName="revoke-keys"
Oct 30 20:47:33.989356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989331    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2ad00ec3-72b1-4f47-9207-646bf9fdd50c" containerName="payload-pre-processing"
Oct 30 20:47:33.989356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989338    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ad00ec3-72b1-4f47-9207-646bf9fdd50c" containerName="payload-pre-processing"
Oct 30 20:47:33.989475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989366    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2a27ab3c-e441-4100-8b5b-e9023c56762b" containerName="payload-processing"
Oct 30 20:47:33.989475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989372    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a27ab3c-e441-4100-8b5b-e9023c56762b" containerName="payload-processing"
Oct 30 20:47:33.989475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989457    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ad00ec3-72b1-4f47-9207-646bf9fdd50c" containerName="payload-pre-processing"
Oct 30 20:47:33.989475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989470    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a27ab3c-e441-4100-8b5b-e9023c56762b" containerName="payload-processing"
Oct 30 20:47:33.989610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.989479    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c47dbd4b-61c4-4944-819b-13d260bb0c28" containerName="revoke-keys"
Oct 30 20:47:33.992255 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.992239    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:33.996823 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.996800    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-53aa47-dockercfg-658pd\""
Oct 30 20:47:33.998032 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:33.998012    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-53aa47\""
Oct 30 20:47:34.005940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.005888    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:34.006603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.006580    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a27ab3c-e441-4100-8b5b-e9023c56762b" path="/var/lib/kubelet/pods/2a27ab3c-e441-4100-8b5b-e9023c56762b/volumes"
Oct 30 20:47:34.007034 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.007018    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ad00ec3-72b1-4f47-9207-646bf9fdd50c" path="/var/lib/kubelet/pods/2ad00ec3-72b1-4f47-9207-646bf9fdd50c/volumes"
Oct 30 20:47:34.007378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.007351    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"]
Oct 30 20:47:34.018613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.018584    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"]
Oct 30 20:47:34.021181 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.021162    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.035440 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.035378    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"]
Oct 30 20:47:34.038931 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.038902    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgsvs\" (UniqueName: \"kubernetes.io/projected/d490b798-a939-4597-b893-212a76b2f589-kube-api-access-pgsvs\") pod \"payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:34.039060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.038970    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d490b798-a939-4597-b893-212a76b2f589-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:34.140040 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.140003    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.140191 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.140049    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgsvs\" (UniqueName: \"kubernetes.io/projected/d490b798-a939-4597-b893-212a76b2f589-kube-api-access-pgsvs\") pod \"payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:34.140191 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.140155    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d490b798-a939-4597-b893-212a76b2f589-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:34.140296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.140279    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvqp4\" (UniqueName: \"kubernetes.io/projected/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-kube-api-access-hvqp4\") pod \"payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.140902 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.140879    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d490b798-a939-4597-b893-212a76b2f589-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:34.154023 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.153998    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgsvs\" (UniqueName: \"kubernetes.io/projected/d490b798-a939-4597-b893-212a76b2f589-kube-api-access-pgsvs\") pod \"payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:34.240944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.240848    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.241137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.240977    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvqp4\" (UniqueName: \"kubernetes.io/projected/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-kube-api-access-hvqp4\") pod \"payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.241578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.241557    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.252647 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.252623    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvqp4\" (UniqueName: \"kubernetes.io/projected/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-kube-api-access-hvqp4\") pod \"payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.307915 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.307876    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:34.335319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.335203    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:34.483733 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.483699    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"]
Oct 30 20:47:34.560047 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.560014    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:34.597365 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.597335    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"]
Oct 30 20:47:34.599334 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:34.599303    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd490b798_a939_4597_b893_212a76b2f589.slice/crio-ae7f9397429492f089e1a6bfd60730f6065a67c01acfa4e065416fe2eafe41cf WatchSource:0}: Error finding container ae7f9397429492f089e1a6bfd60730f6065a67c01acfa4e065416fe2eafe41cf: Status 404 returned error can't find the container with id ae7f9397429492f089e1a6bfd60730f6065a67c01acfa4e065416fe2eafe41cf
Oct 30 20:47:34.630377 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.630350    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"]
Oct 30 20:47:34.631597 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:34.631571    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac26d5f2_d10c_4a5b_ad45_44644b6f776a.slice/crio-2465341eb2f014b799d1f0df5857b1b3bd91377edb2faec30d5f787a7185823d WatchSource:0}: Error finding container 2465341eb2f014b799d1f0df5857b1b3bd91377edb2faec30d5f787a7185823d: Status 404 returned error can't find the container with id 2465341eb2f014b799d1f0df5857b1b3bd91377edb2faec30d5f787a7185823d
Oct 30 20:47:34.662576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.662548    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:34.795648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.795618    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:34.905943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:34.904369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:35.134934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.134901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:35.598970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.598934    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" event={"ID":"ac26d5f2-d10c-4a5b-ad45-44644b6f776a","Type":"ContainerStarted","Data":"a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df"}
Oct 30 20:47:35.599180 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.598990    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" event={"ID":"ac26d5f2-d10c-4a5b-ad45-44644b6f776a","Type":"ContainerStarted","Data":"2465341eb2f014b799d1f0df5857b1b3bd91377edb2faec30d5f787a7185823d"}
Oct 30 20:47:35.599180 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.599087    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:35.600595 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.600567    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" event={"ID":"d490b798-a939-4597-b893-212a76b2f589","Type":"ContainerStarted","Data":"ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68"}
Oct 30 20:47:35.600719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.600614    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:35.600719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.600637    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" event={"ID":"d490b798-a939-4597-b893-212a76b2f589","Type":"ContainerStarted","Data":"ae7f9397429492f089e1a6bfd60730f6065a67c01acfa4e065416fe2eafe41cf"}
Oct 30 20:47:35.620954 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.620906    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" podStartSLOduration=2.620892972 podStartE2EDuration="2.620892972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:35.619222205 +0000 UTC m=+563.733956394" watchObservedRunningTime="2026-10-30 20:47:35.620892972 +0000 UTC m=+563.735627203"
Oct 30 20:47:35.644739 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.644684    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" podStartSLOduration=2.644670707 podStartE2EDuration="2.644670707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:35.642873643 +0000 UTC m=+563.757607836" watchObservedRunningTime="2026-10-30 20:47:35.644670707 +0000 UTC m=+563.759404931"
Oct 30 20:47:35.824205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.824172    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"]
Oct 30 20:47:35.847169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.847130    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"]
Oct 30 20:47:35.899109 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:35.899016    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:36.000664 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.000619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:36.154828 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.154720    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:36.261651 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.261615    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:36.372597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.372565    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:36.637428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.637378    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:36.754874 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.754841    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:36.866431 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.862508    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"]
Oct 30 20:47:36.949916 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:36.949833    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:37.059558 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.059520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:37.243654 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.243573    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:37.339120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.339089    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:37.489444 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.489407    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"]
Oct 30 20:47:37.489709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.489677    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy" containerID="cri-o://80469cc272f51a62318a80a1f22f7965b1ae2c05fd28def08cde1ae8fdc8b687" gracePeriod=30
Oct 30 20:47:37.553719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.553688    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:37.832667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.832584    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:37.943980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:37.943947    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:38.325510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.325477    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:38.418602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.418571    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"]
Oct 30 20:47:38.418841 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.418815    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx" podUID="3fcd9f66-2232-49df-a3fd-e1be5ed69a82" containerName="maas-api" containerID="cri-o://fb777edc9ba6c1867e0707f3c98bd5de0a739839211eafe500d7f8e29c5f5c87" gracePeriod=30
Oct 30 20:47:38.471062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.471024    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:38.623919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.623883    4852 generic.go:356] "Generic (PLEG): container finished" podID="3fcd9f66-2232-49df-a3fd-e1be5ed69a82" containerID="fb777edc9ba6c1867e0707f3c98bd5de0a739839211eafe500d7f8e29c5f5c87" exitCode=0
Oct 30 20:47:38.624087 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.623919    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx" event={"ID":"3fcd9f66-2232-49df-a3fd-e1be5ed69a82","Type":"ContainerDied","Data":"fb777edc9ba6c1867e0707f3c98bd5de0a739839211eafe500d7f8e29c5f5c87"}
Oct 30 20:47:38.669815 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.669789    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:38.694881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.694853    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:38.759842 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.759810    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"]
Oct 30 20:47:38.760373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.760349    4852 cpu_manager.go:414] "containerMap: removing container" podUID="3fcd9f66-2232-49df-a3fd-e1be5ed69a82" containerName="maas-api"
Oct 30 20:47:38.760373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.760373    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fcd9f66-2232-49df-a3fd-e1be5ed69a82" containerName="maas-api"
Oct 30 20:47:38.760573 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.760506    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fcd9f66-2232-49df-a3fd-e1be5ed69a82" containerName="maas-api"
Oct 30 20:47:38.762652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.762632    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.780823 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.780793    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"]
Oct 30 20:47:38.795919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.795889    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-openshift-service-ca\") pod \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") "
Oct 30 20:47:38.796073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.795972    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-metrics-certs\") pod \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") "
Oct 30 20:47:38.796137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.796094    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-maas-api-tls\") pod \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") "
Oct 30 20:47:38.796137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.796127    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-kube-api-access-s55jh\" (UniqueName: \"kubernetes.io/projected/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-kube-api-access-s55jh\") pod \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\" (UID: \"3fcd9f66-2232-49df-a3fd-e1be5ed69a82\") "
Oct 30 20:47:38.796300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.796275    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-openshift-service-ca" pod "3fcd9f66-2232-49df-a3fd-e1be5ed69a82" (UID: "3fcd9f66-2232-49df-a3fd-e1be5ed69a82"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:38.796540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.796518    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:38.798761 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.798732    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-kube-api-access-s55jh" pod "3fcd9f66-2232-49df-a3fd-e1be5ed69a82" (UID: "3fcd9f66-2232-49df-a3fd-e1be5ed69a82"). InnerVolumeSpecName "kube-api-access-s55jh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:38.798871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.798736    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-maas-api-tls" pod "3fcd9f66-2232-49df-a3fd-e1be5ed69a82" (UID: "3fcd9f66-2232-49df-a3fd-e1be5ed69a82"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:47:38.798871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.798766    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-metrics-certs" pod "3fcd9f66-2232-49df-a3fd-e1be5ed69a82" (UID: "3fcd9f66-2232-49df-a3fd-e1be5ed69a82"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:47:38.814546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.814496    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-317a33-openshift-default-dockercfg-9qjvh\""
Oct 30 20:47:38.820555 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.820531    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:38.897750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.897662    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-kube-api-access-tbnnn\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.897750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.897714    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.897968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.897789    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.897968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.897827    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642feace-727f-49a7-8955-b895832d2540-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.897968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.897862    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.898066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.897984    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.898066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.898038    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.898066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.898059    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.898170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.898077    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642feace-727f-49a7-8955-b895832d2540-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.898206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.898168    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:38.898206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.898182    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:38.898206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.898191    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s55jh\" (UniqueName: \"kubernetes.io/projected/3fcd9f66-2232-49df-a3fd-e1be5ed69a82-kube-api-access-s55jh\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:38.920086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.920056    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:38.998944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.998908    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.998953    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999011    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999041    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999069    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642feace-727f-49a7-8955-b895832d2540-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999123    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-kube-api-access-tbnnn\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999175    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999354    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999429    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999447    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999478    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642feace-727f-49a7-8955-b895832d2540-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999500    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:38.999609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:38.999535    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:39.000194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.000168    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642feace-727f-49a7-8955-b895832d2540-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:39.001576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.001555    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:39.001710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.001691    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642feace-727f-49a7-8955-b895832d2540-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:39.008334 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.008313    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:39.008533 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.008516    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-kube-api-access-tbnnn\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:39.076300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.076264    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:39.377499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.377464    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"]
Oct 30 20:47:39.382315 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:39.382277    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642feace_727f_49a7_8955_b895832d2540.slice/crio-b377e09643ec45791c55459bdd815ea96dec80a746b3b1062dd2603e69f07449 WatchSource:0}: Error finding container b377e09643ec45791c55459bdd815ea96dec80a746b3b1062dd2603e69f07449: Status 404 returned error can't find the container with id b377e09643ec45791c55459bdd815ea96dec80a746b3b1062dd2603e69f07449
Oct 30 20:47:39.385381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.385339    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:39.392779 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.385447    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:39.392779 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.385488    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:39.478961 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.478926    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"]
Oct 30 20:47:39.479167 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.479145    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy" containerID="cri-o://0743524eb2adcb8e9ebb5815cd64f3cf2df98fe615b9a268fcea4c98da1a3f5d" gracePeriod=30
Oct 30 20:47:39.485003 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.484940    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n" podUID="83015a0c-51f7-4382-8b4d-2b5d24f95b82" containerName="payload-pre-processing" containerID="cri-o://b24eb6f0216b7f885856e0324d47875f01a58deead151765da15a7816fc7bc37" gracePeriod=25
Oct 30 20:47:39.596411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.593651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:39.637284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.634567    4852 generic.go:356] "Generic (PLEG): container finished" podID="83015a0c-51f7-4382-8b4d-2b5d24f95b82" containerID="b24eb6f0216b7f885856e0324d47875f01a58deead151765da15a7816fc7bc37" exitCode=0
Oct 30 20:47:39.637284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.634775    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n" event={"ID":"83015a0c-51f7-4382-8b4d-2b5d24f95b82","Type":"ContainerDied","Data":"b24eb6f0216b7f885856e0324d47875f01a58deead151765da15a7816fc7bc37"}
Oct 30 20:47:39.637284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.636598    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx" event={"ID":"3fcd9f66-2232-49df-a3fd-e1be5ed69a82","Type":"ContainerDied","Data":"4464129619e8eea65f5be3a4f49af585e0f6bda9dbfc7960f3e85373dc4daddb"}
Oct 30 20:47:39.637284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.636652    4852 scope.go:122] "RemoveContainer" containerID="fb777edc9ba6c1867e0707f3c98bd5de0a739839211eafe500d7f8e29c5f5c87"
Oct 30 20:47:39.637284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.636814    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"
Oct 30 20:47:39.643434 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.643135    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" event={"ID":"642feace-727f-49a7-8955-b895832d2540","Type":"ContainerStarted","Data":"836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447"}
Oct 30 20:47:39.643434 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.643179    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" event={"ID":"642feace-727f-49a7-8955-b895832d2540","Type":"ContainerStarted","Data":"b377e09643ec45791c55459bdd815ea96dec80a746b3b1062dd2603e69f07449"}
Oct 30 20:47:39.669415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.669138    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podStartSLOduration=1.669118941 podStartE2EDuration="1.669118941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:39.667983507 +0000 UTC m=+567.782717711" watchObservedRunningTime="2026-10-30 20:47:39.669118941 +0000 UTC m=+567.783853133"
Oct 30 20:47:39.693282 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.693247    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"]
Oct 30 20:47:39.700585 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.700340    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b16c21be-78954c6df7-x7gpx"]
Oct 30 20:47:39.756291 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.756266    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:47:39.909017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.908928    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83015a0c-51f7-4382-8b4d-2b5d24f95b82-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83015a0c-51f7-4382-8b4d-2b5d24f95b82-plugins-config-volume\") pod \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") "
Oct 30 20:47:39.909017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.909005    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83015a0c-51f7-4382-8b4d-2b5d24f95b82-kube-api-access-9zmxw\" (UniqueName: \"kubernetes.io/projected/83015a0c-51f7-4382-8b4d-2b5d24f95b82-kube-api-access-9zmxw\") pod \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\" (UID: \"83015a0c-51f7-4382-8b4d-2b5d24f95b82\") "
Oct 30 20:47:39.909354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.909313    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83015a0c-51f7-4382-8b4d-2b5d24f95b82-plugins-config-volume" pod "83015a0c-51f7-4382-8b4d-2b5d24f95b82" (UID: "83015a0c-51f7-4382-8b4d-2b5d24f95b82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:39.909541 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.909447    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83015a0c-51f7-4382-8b4d-2b5d24f95b82-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:39.911208 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.911175    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83015a0c-51f7-4382-8b4d-2b5d24f95b82-kube-api-access-9zmxw" pod "83015a0c-51f7-4382-8b4d-2b5d24f95b82" (UID: "83015a0c-51f7-4382-8b4d-2b5d24f95b82"). InnerVolumeSpecName "kube-api-access-9zmxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:39.937101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:39.937065    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:40.003534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.003493    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fcd9f66-2232-49df-a3fd-e1be5ed69a82" path="/var/lib/kubelet/pods/3fcd9f66-2232-49df-a3fd-e1be5ed69a82/volumes"
Oct 30 20:47:40.009932 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.009903    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zmxw\" (UniqueName: \"kubernetes.io/projected/83015a0c-51f7-4382-8b4d-2b5d24f95b82-kube-api-access-9zmxw\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:40.061033 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.060998    4852 patch_prober.go:29] interesting pod/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:47:40.061204 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.061059    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:47:40.076928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.076904    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:40.081624 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.081603    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:40.292379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.292342    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:40.651780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.651700    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"
Oct 30 20:47:40.651780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.651738    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n" event={"ID":"83015a0c-51f7-4382-8b4d-2b5d24f95b82","Type":"ContainerDied","Data":"6a5c44aa8f153c1067e3c89d7a1f9f1028e54723a4dd0444feae826f494b4ac3"}
Oct 30 20:47:40.652272 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.651801    4852 scope.go:122] "RemoveContainer" containerID="b24eb6f0216b7f885856e0324d47875f01a58deead151765da15a7816fc7bc37"
Oct 30 20:47:40.652272 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.652153    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:40.653228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.653214    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:40.720773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.720743    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"]
Oct 30 20:47:40.726070 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.726040    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b16c21be-7549f65d9f-pxz4n"]
Oct 30 20:47:40.728378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.728345    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:40.821609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.821574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:40.825485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.825453    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx" podUID="28437a12-dc09-467d-ac94-34172643e99e" containerName="payload-processing" containerID="cri-o://9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341" gracePeriod=25
Oct 30 20:47:40.847960 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.847924    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d" podUID="35ebbd3a-8954-455d-96ec-c159377f315b" containerName="payload-pre-processing" containerID="cri-o://f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64" gracePeriod=25
Oct 30 20:47:40.921346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:40.921314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:41.097973 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.097941    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"]
Oct 30 20:47:41.104015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.103986    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-67598f1d-4k28j"]
Oct 30 20:47:41.112095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.112069    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:41.118923 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.118897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:41.133206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.133180    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:41.205504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.205420    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:41.223600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.223566    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28437a12-dc09-467d-ac94-34172643e99e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28437a12-dc09-467d-ac94-34172643e99e-plugins-config-volume\") pod \"28437a12-dc09-467d-ac94-34172643e99e\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") "
Oct 30 20:47:41.223926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.223618    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35ebbd3a-8954-455d-96ec-c159377f315b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ebbd3a-8954-455d-96ec-c159377f315b-plugins-config-volume\") pod \"35ebbd3a-8954-455d-96ec-c159377f315b\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") "
Oct 30 20:47:41.223926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.223821    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35ebbd3a-8954-455d-96ec-c159377f315b-kube-api-access-lvffm\" (UniqueName: \"kubernetes.io/projected/35ebbd3a-8954-455d-96ec-c159377f315b-kube-api-access-lvffm\") pod \"35ebbd3a-8954-455d-96ec-c159377f315b\" (UID: \"35ebbd3a-8954-455d-96ec-c159377f315b\") "
Oct 30 20:47:41.223926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.223887    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28437a12-dc09-467d-ac94-34172643e99e-kube-api-access-sfv84\" (UniqueName: \"kubernetes.io/projected/28437a12-dc09-467d-ac94-34172643e99e-kube-api-access-sfv84\") pod \"28437a12-dc09-467d-ac94-34172643e99e\" (UID: \"28437a12-dc09-467d-ac94-34172643e99e\") "
Oct 30 20:47:41.224182 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.224153    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35ebbd3a-8954-455d-96ec-c159377f315b-plugins-config-volume" pod "35ebbd3a-8954-455d-96ec-c159377f315b" (UID: "35ebbd3a-8954-455d-96ec-c159377f315b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:41.224300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.224274    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ebbd3a-8954-455d-96ec-c159377f315b-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:41.224563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.224543    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28437a12-dc09-467d-ac94-34172643e99e-plugins-config-volume" pod "28437a12-dc09-467d-ac94-34172643e99e" (UID: "28437a12-dc09-467d-ac94-34172643e99e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:41.226094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.226072    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35ebbd3a-8954-455d-96ec-c159377f315b-kube-api-access-lvffm" pod "35ebbd3a-8954-455d-96ec-c159377f315b" (UID: "35ebbd3a-8954-455d-96ec-c159377f315b"). InnerVolumeSpecName "kube-api-access-lvffm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:41.226381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.226357    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28437a12-dc09-467d-ac94-34172643e99e-kube-api-access-sfv84" pod "28437a12-dc09-467d-ac94-34172643e99e" (UID: "28437a12-dc09-467d-ac94-34172643e99e"). InnerVolumeSpecName "kube-api-access-sfv84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:41.252399 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.252360    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"]
Oct 30 20:47:41.252656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.252611    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy" containerID="cri-o://f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1" gracePeriod=30
Oct 30 20:47:41.306927 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.306893    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:41.325382 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.325353    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvffm\" (UniqueName: \"kubernetes.io/projected/35ebbd3a-8954-455d-96ec-c159377f315b-kube-api-access-lvffm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:41.325526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.325381    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfv84\" (UniqueName: \"kubernetes.io/projected/28437a12-dc09-467d-ac94-34172643e99e-kube-api-access-sfv84\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:41.325526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.325423    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28437a12-dc09-467d-ac94-34172643e99e-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:41.658680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.658641    4852 generic.go:356] "Generic (PLEG): container finished" podID="35ebbd3a-8954-455d-96ec-c159377f315b" containerID="f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64" exitCode=0
Oct 30 20:47:41.659112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.658694    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"
Oct 30 20:47:41.659112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.658721    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d" event={"ID":"35ebbd3a-8954-455d-96ec-c159377f315b","Type":"ContainerDied","Data":"f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64"}
Oct 30 20:47:41.659112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.658774    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d" event={"ID":"35ebbd3a-8954-455d-96ec-c159377f315b","Type":"ContainerDied","Data":"0a84d6e38fe7c3323e21b94582e7959bc7d80400dbb525cfe6f9d4e2e0a3f0c0"}
Oct 30 20:47:41.659112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.658803    4852 scope.go:122] "RemoveContainer" containerID="f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64"
Oct 30 20:47:41.661374 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.661350    4852 generic.go:356] "Generic (PLEG): container finished" podID="28437a12-dc09-467d-ac94-34172643e99e" containerID="9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341" exitCode=0
Oct 30 20:47:41.661529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.661431    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"
Oct 30 20:47:41.661529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.661439    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx" event={"ID":"28437a12-dc09-467d-ac94-34172643e99e","Type":"ContainerDied","Data":"9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341"}
Oct 30 20:47:41.661529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.661507    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx" event={"ID":"28437a12-dc09-467d-ac94-34172643e99e","Type":"ContainerDied","Data":"b8783ddbe6a9d583beefcee362a567f6741ce76264f6569364f82179a8ecfe08"}
Oct 30 20:47:41.670508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.670484    4852 scope.go:122] "RemoveContainer" containerID="f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64"
Oct 30 20:47:41.670811 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:41.670792    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64\": container with ID starting with f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64 not found: ID does not exist" containerID="f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64"
Oct 30 20:47:41.670861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.670819    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64"} err="failed to get container status \"f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64\": rpc error: code = NotFound desc = could not find container \"f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64\": container with ID starting with f402abe68a6abea1eeecbd6b38d4ffea69b61941b99c8b5ac2c3f2e5e855bd64 not found: ID does not exist"
Oct 30 20:47:41.670861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.670835    4852 scope.go:122] "RemoveContainer" containerID="9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341"
Oct 30 20:47:41.685885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.685847    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"]
Oct 30 20:47:41.686947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.686920    4852 scope.go:122] "RemoveContainer" containerID="9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341"
Oct 30 20:47:41.687325 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:41.687302    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341\": container with ID starting with 9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341 not found: ID does not exist" containerID="9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341"
Oct 30 20:47:41.687487 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.687331    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341"} err="failed to get container status \"9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341\": rpc error: code = NotFound desc = could not find container \"9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341\": container with ID starting with 9cf9d1e038df64c84b1c1208ced04498226b2cde8e27a44af8c1a6c682869341 not found: ID does not exist"
Oct 30 20:47:41.691002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.690979    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-eaf151e7-777b69d7-vkp6d"]
Oct 30 20:47:41.708065 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.708027    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"]
Oct 30 20:47:41.714396 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.714357    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-eaf151e7-5996b6df49-sv2qx"]
Oct 30 20:47:41.863244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.863175    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv" podUID="f10d8fa0-b207-49db-86ad-a523b020c31e" containerName="payload-processing" containerID="cri-o://675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895" gracePeriod=25
Oct 30 20:47:41.946147 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:41.946066    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:42.004178 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.004144    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28437a12-dc09-467d-ac94-34172643e99e" path="/var/lib/kubelet/pods/28437a12-dc09-467d-ac94-34172643e99e/volumes"
Oct 30 20:47:42.004744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.004725    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35ebbd3a-8954-455d-96ec-c159377f315b" path="/var/lib/kubelet/pods/35ebbd3a-8954-455d-96ec-c159377f315b/volumes"
Oct 30 20:47:42.005244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.005225    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83015a0c-51f7-4382-8b4d-2b5d24f95b82" path="/var/lib/kubelet/pods/83015a0c-51f7-4382-8b4d-2b5d24f95b82/volumes"
Oct 30 20:47:42.006236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.006200    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c47dbd4b-61c4-4944-819b-13d260bb0c28" path="/var/lib/kubelet/pods/c47dbd4b-61c4-4944-819b-13d260bb0c28/volumes"
Oct 30 20:47:42.100613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.100580    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:42.124713 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.124690    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:47:42.236281 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.236170    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f10d8fa0-b207-49db-86ad-a523b020c31e-kube-api-access-cb2bx\" (UniqueName: \"kubernetes.io/projected/f10d8fa0-b207-49db-86ad-a523b020c31e-kube-api-access-cb2bx\") pod \"f10d8fa0-b207-49db-86ad-a523b020c31e\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") "
Oct 30 20:47:42.236281 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.236231    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f10d8fa0-b207-49db-86ad-a523b020c31e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f10d8fa0-b207-49db-86ad-a523b020c31e-plugins-config-volume\") pod \"f10d8fa0-b207-49db-86ad-a523b020c31e\" (UID: \"f10d8fa0-b207-49db-86ad-a523b020c31e\") "
Oct 30 20:47:42.236672 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.236634    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f10d8fa0-b207-49db-86ad-a523b020c31e-plugins-config-volume" pod "f10d8fa0-b207-49db-86ad-a523b020c31e" (UID: "f10d8fa0-b207-49db-86ad-a523b020c31e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:42.236746 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.236723    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f10d8fa0-b207-49db-86ad-a523b020c31e-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.238322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.238297    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f10d8fa0-b207-49db-86ad-a523b020c31e-kube-api-access-cb2bx" pod "f10d8fa0-b207-49db-86ad-a523b020c31e" (UID: "f10d8fa0-b207-49db-86ad-a523b020c31e"). InnerVolumeSpecName "kube-api-access-cb2bx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:42.278048 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.278011    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:42.337872 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.337831    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb2bx\" (UniqueName: \"kubernetes.io/projected/f10d8fa0-b207-49db-86ad-a523b020c31e-kube-api-access-cb2bx\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.533205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.533177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:42.585472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585440    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"]
Oct 30 20:47:42.585919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585900    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f10d8fa0-b207-49db-86ad-a523b020c31e" containerName="payload-processing"
Oct 30 20:47:42.585919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585917    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f10d8fa0-b207-49db-86ad-a523b020c31e" containerName="payload-processing"
Oct 30 20:47:42.586096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585937    4852 cpu_manager.go:414] "containerMap: removing container" podUID="28437a12-dc09-467d-ac94-34172643e99e" containerName="payload-processing"
Oct 30 20:47:42.586096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585945    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28437a12-dc09-467d-ac94-34172643e99e" containerName="payload-processing"
Oct 30 20:47:42.586096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585961    4852 cpu_manager.go:414] "containerMap: removing container" podUID="83015a0c-51f7-4382-8b4d-2b5d24f95b82" containerName="payload-pre-processing"
Oct 30 20:47:42.586096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585971    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83015a0c-51f7-4382-8b4d-2b5d24f95b82" containerName="payload-pre-processing"
Oct 30 20:47:42.586096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.585993    4852 cpu_manager.go:414] "containerMap: removing container" podUID="35ebbd3a-8954-455d-96ec-c159377f315b" containerName="payload-pre-processing"
Oct 30 20:47:42.586096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.586001    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35ebbd3a-8954-455d-96ec-c159377f315b" containerName="payload-pre-processing"
Oct 30 20:47:42.586096 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.586092    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="28437a12-dc09-467d-ac94-34172643e99e" containerName="payload-processing"
Oct 30 20:47:42.586315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.586103    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="83015a0c-51f7-4382-8b4d-2b5d24f95b82" containerName="payload-pre-processing"
Oct 30 20:47:42.586315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.586111    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="35ebbd3a-8954-455d-96ec-c159377f315b" containerName="payload-pre-processing"
Oct 30 20:47:42.586315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.586122    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f10d8fa0-b207-49db-86ad-a523b020c31e" containerName="payload-processing"
Oct 30 20:47:42.588090 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.588072    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.598049 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.598026    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-8e4acb-openshift-default-dockercfg-w4qvm\""
Oct 30 20:47:42.609438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.609405    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"]
Oct 30 20:47:42.662659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.662529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:42.670753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.670729    4852 generic.go:356] "Generic (PLEG): container finished" podID="f10d8fa0-b207-49db-86ad-a523b020c31e" containerID="675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895" exitCode=0
Oct 30 20:47:42.671025 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.670767    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv" event={"ID":"f10d8fa0-b207-49db-86ad-a523b020c31e","Type":"ContainerDied","Data":"675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895"}
Oct 30 20:47:42.671144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.671064    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv" event={"ID":"f10d8fa0-b207-49db-86ad-a523b020c31e","Type":"ContainerDied","Data":"41df407a1b04d31e18f085e2cc9b1984c3654c09f98bf35434987a59d2215735"}
Oct 30 20:47:42.671144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.671092    4852 scope.go:122] "RemoveContainer" containerID="675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895"
Oct 30 20:47:42.671144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.670831    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"
Oct 30 20:47:42.680273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.680247    4852 generic.go:356] "Generic (PLEG): container finished" podID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerID="80469cc272f51a62318a80a1f22f7965b1ae2c05fd28def08cde1ae8fdc8b687" exitCode=0
Oct 30 20:47:42.680379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.680326    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" event={"ID":"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b","Type":"ContainerDied","Data":"80469cc272f51a62318a80a1f22f7965b1ae2c05fd28def08cde1ae8fdc8b687"}
Oct 30 20:47:42.687268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.687235    4852 scope.go:122] "RemoveContainer" containerID="675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895"
Oct 30 20:47:42.687614 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:42.687590    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895\": container with ID starting with 675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895 not found: ID does not exist" containerID="675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895"
Oct 30 20:47:42.687724 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.687625    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895"} err="failed to get container status \"675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895\": rpc error: code = NotFound desc = could not find container \"675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895\": container with ID starting with 675d6fcbbc9e2805f0450cd959e71741861376ce34e08f23a98479d24b544895 not found: ID does not exist"
Oct 30 20:47:42.687724 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.687646    4852 scope.go:122] "RemoveContainer" containerID="96c7dfe133eeaf37813d4874e0e6da94abd1bdb196eff07392f06b73de1ea0ec"
Oct 30 20:47:42.697178 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.697153    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"]
Oct 30 20:47:42.706612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.706548    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67598f1d-6db88fd7dc-xrmbv"]
Oct 30 20:47:42.736241 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.736216    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:42.741607 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741574    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.741798 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741684    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.741798 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741715    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hc8l\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-kube-api-access-8hc8l\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.741798 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741742    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.741798 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741774    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.742017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741828    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.742017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741871    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.742017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741941    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.742017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.741966    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.781705 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.781662    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:42.842542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842514    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-podinfo\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.842690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842585    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-socket\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.842690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842644    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-kube-api-access-8vfz5\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-kube-api-access-8vfz5\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.842690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842676    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-credential-socket\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.842859 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842726    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-envoy\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.842859 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842794    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-data\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.842859 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842823    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istiod-ca-cert\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.843006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842870    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-certs\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.843006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842899    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-token\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-token\") pod \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\" (UID: \"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b\") "
Oct 30 20:47:42.843006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.842902    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-socket" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:42.843189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843073    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843103    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hc8l\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-kube-api-access-8hc8l\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843124    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-data" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:42.843189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843134    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843142    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-certs" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:42.843449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843206    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843215    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istiod-ca-cert" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:42.843449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843307    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843372    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843439    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843465    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843470    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843651    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.843769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843723    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.843769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843740    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.843769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843754    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.843769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843768    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.844254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843807    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.844254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.843963    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-credential-socket" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:42.844254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.844210    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.844870 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.844846    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.844944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.844916    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.845320 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.845301    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-kube-api-access-8vfz5" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "kube-api-access-8vfz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:42.845787 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.845756    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-token" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:42.845894 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.845843    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-envoy" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:42.846930 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.846907    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.846930 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.846922    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-podinfo" pod "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" (UID: "20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:47:42.847822 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.847802    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.854338 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.854315    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.854790 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.854754    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hc8l\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-kube-api-access-8hc8l\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.900740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.900709    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:42.944317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.944284    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.944317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.944314    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vfz5\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-kube-api-access-8vfz5\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.944317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.944325    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.944573 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.944334    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:42.944573 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:42.944342    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:43.040820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.040726    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:43.184763 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.184735    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"]
Oct 30 20:47:43.186354 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:43.186328    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc925dcc8_57c5_47aa_9dd4_a011fadf74da.slice/crio-5cdb139b0a34d7624f0c087128c517bdf6ef5ad33f5afc9f12f2b5437a223e02 WatchSource:0}: Error finding container 5cdb139b0a34d7624f0c087128c517bdf6ef5ad33f5afc9f12f2b5437a223e02: Status 404 returned error can't find the container with id 5cdb139b0a34d7624f0c087128c517bdf6ef5ad33f5afc9f12f2b5437a223e02
Oct 30 20:47:43.188734 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.188702    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:43.188791 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.188770    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:43.188834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.188798    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:43.504756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.504656    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:43.621962 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.621921    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:43.687546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.687509    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r" event={"ID":"20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b","Type":"ContainerDied","Data":"371c0259308fd356136aa6afc08d686635cc440065bb87ac644a3d5100b06177"}
Oct 30 20:47:43.687955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.687567    4852 scope.go:122] "RemoveContainer" containerID="80469cc272f51a62318a80a1f22f7965b1ae2c05fd28def08cde1ae8fdc8b687"
Oct 30 20:47:43.687955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.687716    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"
Oct 30 20:47:43.691583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.691553    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" event={"ID":"c925dcc8-57c5-47aa-9dd4-a011fadf74da","Type":"ContainerStarted","Data":"2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1"}
Oct 30 20:47:43.691694 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.691612    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" event={"ID":"c925dcc8-57c5-47aa-9dd4-a011fadf74da","Type":"ContainerStarted","Data":"5cdb139b0a34d7624f0c087128c517bdf6ef5ad33f5afc9f12f2b5437a223e02"}
Oct 30 20:47:43.696284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.696251    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"]
Oct 30 20:47:43.697072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.697051    4852 cpu_manager.go:414] "containerMap: removing container" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy"
Oct 30 20:47:43.697149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.697072    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy"
Oct 30 20:47:43.697149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.697092    4852 cpu_manager.go:414] "containerMap: removing container" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy"
Oct 30 20:47:43.697149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.697099    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy"
Oct 30 20:47:43.697299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.697197    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy"
Oct 30 20:47:43.700678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.700656    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:43.708844 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.707461    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-317a33\""
Oct 30 20:47:43.708844 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.707622    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-317a33-dockercfg-rjff2\""
Oct 30 20:47:43.711548 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.711523    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"]
Oct 30 20:47:43.711802 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.711773    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:43.716606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.716560    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podStartSLOduration=1.716547431 podStartE2EDuration="1.716547431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:43.714733191 +0000 UTC m=+571.829467382" watchObservedRunningTime="2026-10-30 20:47:43.716547431 +0000 UTC m=+571.831281621"
Oct 30 20:47:43.731045 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.731009    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"]
Oct 30 20:47:43.741779 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.741746    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-eaf151e7-openshift-default-668cb9db45-jqz2r"]
Oct 30 20:47:43.745610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.745583    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"]
Oct 30 20:47:43.746321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.746302    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" containerName="istio-proxy"
Oct 30 20:47:43.749268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.749249    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:43.757000 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.756926    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"]
Oct 30 20:47:43.851967 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.851930    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b319c-7082-40df-b5ef-b3eb6b43ad90-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:43.852118 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.851991    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn8xw\" (UniqueName: \"kubernetes.io/projected/d67b319c-7082-40df-b5ef-b3eb6b43ad90-kube-api-access-kn8xw\") pod \"payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:43.852192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.852146    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t972h\" (UniqueName: \"kubernetes.io/projected/979b5e79-af17-48e2-969a-96a6ef1b4a37-kube-api-access-t972h\") pod \"payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:43.852369 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.852344    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/979b5e79-af17-48e2-969a-96a6ef1b4a37-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:43.902425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.900820    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:43.953876 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.953846    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t972h\" (UniqueName: \"kubernetes.io/projected/979b5e79-af17-48e2-969a-96a6ef1b4a37-kube-api-access-t972h\") pod \"payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:43.954022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.953936    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/979b5e79-af17-48e2-969a-96a6ef1b4a37-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:43.954022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.953971    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b319c-7082-40df-b5ef-b3eb6b43ad90-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:43.954121 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.954100    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn8xw\" (UniqueName: \"kubernetes.io/projected/d67b319c-7082-40df-b5ef-b3eb6b43ad90-kube-api-access-kn8xw\") pod \"payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:43.954920 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.954897    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b319c-7082-40df-b5ef-b3eb6b43ad90-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:43.955017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.954897    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/979b5e79-af17-48e2-969a-96a6ef1b4a37-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:43.964974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.964940    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t972h\" (UniqueName: \"kubernetes.io/projected/979b5e79-af17-48e2-969a-96a6ef1b4a37-kube-api-access-t972h\") pod \"payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:43.965150 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:43.965002    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn8xw\" (UniqueName: \"kubernetes.io/projected/d67b319c-7082-40df-b5ef-b3eb6b43ad90-kube-api-access-kn8xw\") pod \"payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:44.002598 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.002567    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:44.003345 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.003317    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b" path="/var/lib/kubelet/pods/20cf0a5b-6b0c-4b84-b772-80e6b7edaf7b/volumes"
Oct 30 20:47:44.003850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.003829    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f10d8fa0-b207-49db-86ad-a523b020c31e" path="/var/lib/kubelet/pods/f10d8fa0-b207-49db-86ad-a523b020c31e/volumes"
Oct 30 20:47:44.017246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.017177    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:44.067360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.067120    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:44.103904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.103864    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:44.297516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.297491    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"]
Oct 30 20:47:44.299288 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:44.299259    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67b319c_7082_40df_b5ef_b3eb6b43ad90.slice/crio-91d3ec5162441e98698f511611aff2163e76ff876122053093062945652f974d WatchSource:0}: Error finding container 91d3ec5162441e98698f511611aff2163e76ff876122053093062945652f974d: Status 404 returned error can't find the container with id 91d3ec5162441e98698f511611aff2163e76ff876122053093062945652f974d
Oct 30 20:47:44.359151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.359125    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"]
Oct 30 20:47:44.360843 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:44.360800    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979b5e79_af17_48e2_969a_96a6ef1b4a37.slice/crio-2161187690efb75ed776e747b2caaec54310acaab541792ff4617a3e1181024e WatchSource:0}: Error finding container 2161187690efb75ed776e747b2caaec54310acaab541792ff4617a3e1181024e: Status 404 returned error can't find the container with id 2161187690efb75ed776e747b2caaec54310acaab541792ff4617a3e1181024e
Oct 30 20:47:44.701181 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.701140    4852 generic.go:356] "Generic (PLEG): container finished" podID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerID="0743524eb2adcb8e9ebb5815cd64f3cf2df98fe615b9a268fcea4c98da1a3f5d" exitCode=0
Oct 30 20:47:44.701741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.701221    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" event={"ID":"fd263061-7b8f-49b4-b1bf-9c1f7c579af7","Type":"ContainerDied","Data":"0743524eb2adcb8e9ebb5815cd64f3cf2df98fe615b9a268fcea4c98da1a3f5d"}
Oct 30 20:47:44.701741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.701336    4852 scope.go:122] "RemoveContainer" containerID="019f4c6a3c5ae1581cfc55b4718035df880f0b6c9529d3abb98ae924610bb85a"
Oct 30 20:47:44.704062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.704033    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" event={"ID":"979b5e79-af17-48e2-969a-96a6ef1b4a37","Type":"ContainerStarted","Data":"7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964"}
Oct 30 20:47:44.704180 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.704086    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" event={"ID":"979b5e79-af17-48e2-969a-96a6ef1b4a37","Type":"ContainerStarted","Data":"2161187690efb75ed776e747b2caaec54310acaab541792ff4617a3e1181024e"}
Oct 30 20:47:44.704254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.704228    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:44.705944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.705917    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" event={"ID":"d67b319c-7082-40df-b5ef-b3eb6b43ad90","Type":"ContainerStarted","Data":"25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a"}
Oct 30 20:47:44.706086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.705965    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" event={"ID":"d67b319c-7082-40df-b5ef-b3eb6b43ad90","Type":"ContainerStarted","Data":"91d3ec5162441e98698f511611aff2163e76ff876122053093062945652f974d"}
Oct 30 20:47:44.706086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.706020    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:44.725296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.725268    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:47:44.725896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.725857    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" podStartSLOduration=1.725844091 podStartE2EDuration="1.725844091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:44.72307423 +0000 UTC m=+572.837808420" watchObservedRunningTime="2026-10-30 20:47:44.725844091 +0000 UTC m=+572.840578281"
Oct 30 20:47:44.742148 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.742077    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" podStartSLOduration=1.74206467 podStartE2EDuration="1.74206467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:44.740730481 +0000 UTC m=+572.855464670" watchObservedRunningTime="2026-10-30 20:47:44.74206467 +0000 UTC m=+572.856798861"
Oct 30 20:47:44.862994 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.862706    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-kube-api-access-vrc4f\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-kube-api-access-vrc4f\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.863649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.863547    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istiod-ca-cert\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.864020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.863870    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-socket\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.864020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.863920    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-credential-socket\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.864020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.863969    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-envoy\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.864020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.864007    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istiod-ca-cert" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:44.864632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.864321    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-socket" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:44.864632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.864353    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-podinfo\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.864632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.864479    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-certs\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.864632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.864560    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-token\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-token\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.864632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.864625    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-credential-socket" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:44.864953 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.864672    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-data\") pod \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\" (UID: \"fd263061-7b8f-49b4-b1bf-9c1f7c579af7\") "
Oct 30 20:47:44.866306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.865534    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.866306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.865575    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.866306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.865590    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.866306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.865946    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-certs" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:44.870269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.870186    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-kube-api-access-vrc4f" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "kube-api-access-vrc4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:44.870269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.870211    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-data" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:44.871147 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.871112    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-envoy" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:44.871460 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.871435    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-token" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:44.871557 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.871462    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-podinfo" pod "fd263061-7b8f-49b4-b1bf-9c1f7c579af7" (UID: "fd263061-7b8f-49b4-b1bf-9c1f7c579af7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:47:44.889749 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.889716    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"]
Oct 30 20:47:44.890340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.890319    4852 cpu_manager.go:414] "containerMap: removing container" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy"
Oct 30 20:47:44.890458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.890342    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy"
Oct 30 20:47:44.890458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.890368    4852 cpu_manager.go:414] "containerMap: removing container" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy"
Oct 30 20:47:44.890458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.890376    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy"
Oct 30 20:47:44.890602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.890528    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy"
Oct 30 20:47:44.894330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.894306    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:44.905851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.905454    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 20:47:44.905851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.905550    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 20:47:44.908571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.908528    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:44.909472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.909451    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"]
Oct 30 20:47:44.911128 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.910779    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bbd9l\""
Oct 30 20:47:44.911128 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.910841    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-317a33-serving-cert\""
Oct 30 20:47:44.911128 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.910909    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 20:47:44.943472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.943437    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:44.966180 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966149    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-metrics-certs\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:44.966335 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966185    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc65f10-d54b-4238-82ff-71b4a623b7ca-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:44.966335 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966212    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnfn8\" (UniqueName: \"kubernetes.io/projected/cbc65f10-d54b-4238-82ff-71b4a623b7ca-kube-api-access-fnfn8\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:44.966470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966357    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-maas-api-tls\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:44.966506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966485    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.966506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966498    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.966571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966511    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.966571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966525    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.966571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966534    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrc4f\" (UniqueName: \"kubernetes.io/projected/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-kube-api-access-vrc4f\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:44.966571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:44.966543    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd263061-7b8f-49b4-b1bf-9c1f7c579af7-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:45.021321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.021289    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:45.067113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.067076    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-metrics-certs\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.067113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.067114    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc65f10-d54b-4238-82ff-71b4a623b7ca-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.067327 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.067141    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfn8\" (UniqueName: \"kubernetes.io/projected/cbc65f10-d54b-4238-82ff-71b4a623b7ca-kube-api-access-fnfn8\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.067327 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.067184    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-maas-api-tls\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.067867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.067844    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc65f10-d54b-4238-82ff-71b4a623b7ca-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.069651 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.069630    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-metrics-certs\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.069748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.069717    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-maas-api-tls\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.077315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.077289    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnfn8\" (UniqueName: \"kubernetes.io/projected/cbc65f10-d54b-4238-82ff-71b4a623b7ca-kube-api-access-fnfn8\") pod \"maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.122012 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.121937    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:45.215437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.215382    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:45.421242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.421158    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:45.495436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.495408    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"]
Oct 30 20:47:45.496799 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:45.496758    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc65f10_d54b_4238_82ff_71b4a623b7ca.slice/crio-6caaac5a24a72b407ff6029407708ca7305d00ab1b3e7227db40544d7cde14a5 WatchSource:0}: Error finding container 6caaac5a24a72b407ff6029407708ca7305d00ab1b3e7227db40544d7cde14a5: Status 404 returned error can't find the container with id 6caaac5a24a72b407ff6029407708ca7305d00ab1b3e7227db40544d7cde14a5
Oct 30 20:47:45.551578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.551543    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:45.570765 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.570723    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"]
Oct 30 20:47:45.571322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.571302    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" containerName="istio-proxy"
Oct 30 20:47:45.575570 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.575546    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:45.587370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.587342    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"]
Oct 30 20:47:45.588686 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.588650    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-bc102e\""
Oct 30 20:47:45.588805 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.588790    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-bc102e-dockercfg-pvrvc\""
Oct 30 20:47:45.673547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.673457    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g476\" (UniqueName: \"kubernetes.io/projected/c70bde44-f354-4a89-ac63-af5e6300f8fc-kube-api-access-6g476\") pod \"payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:45.673547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.673509    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c70bde44-f354-4a89-ac63-af5e6300f8fc-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:45.715214 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.715183    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"
Oct 30 20:47:45.715658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.715222    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6" event={"ID":"fd263061-7b8f-49b4-b1bf-9c1f7c579af7","Type":"ContainerDied","Data":"175c82448772f6d251120b7f98e4782669364b5a0a30d37e4971ac412419f337"}
Oct 30 20:47:45.715658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.715286    4852 scope.go:122] "RemoveContainer" containerID="0743524eb2adcb8e9ebb5815cd64f3cf2df98fe615b9a268fcea4c98da1a3f5d"
Oct 30 20:47:45.716821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.716785    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r" event={"ID":"cbc65f10-d54b-4238-82ff-71b4a623b7ca","Type":"ContainerStarted","Data":"6caaac5a24a72b407ff6029407708ca7305d00ab1b3e7227db40544d7cde14a5"}
Oct 30 20:47:45.717090 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.717067    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:45.718368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.718346    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:47:45.774938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.774899    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g476\" (UniqueName: \"kubernetes.io/projected/c70bde44-f354-4a89-ac63-af5e6300f8fc-kube-api-access-6g476\") pod \"payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:45.775155 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.774970    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c70bde44-f354-4a89-ac63-af5e6300f8fc-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:45.775857 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.775832    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c70bde44-f354-4a89-ac63-af5e6300f8fc-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:45.800142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.800110    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g476\" (UniqueName: \"kubernetes.io/projected/c70bde44-f354-4a89-ac63-af5e6300f8fc-kube-api-access-6g476\") pod \"payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:45.804517 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.804489    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"]
Oct 30 20:47:45.834748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.834715    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b16c21be-openshift-default-8598bb4d8f-tbpp6"]
Oct 30 20:47:45.889717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:45.889681    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:46.003942 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.003910    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd263061-7b8f-49b4-b1bf-9c1f7c579af7" path="/var/lib/kubelet/pods/fd263061-7b8f-49b4-b1bf-9c1f7c579af7/volumes"
Oct 30 20:47:46.098279 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.098244    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:46.192594 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.192565    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"]
Oct 30 20:47:46.194590 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:46.194563    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70bde44_f354_4a89_ac63_af5e6300f8fc.slice/crio-fe4fb2bffb4b2dc844eefef70f08c68b762966eac6d85fc49d893485fa41960e WatchSource:0}: Error finding container fe4fb2bffb4b2dc844eefef70f08c68b762966eac6d85fc49d893485fa41960e: Status 404 returned error can't find the container with id fe4fb2bffb4b2dc844eefef70f08c68b762966eac6d85fc49d893485fa41960e
Oct 30 20:47:46.214516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.214484    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:46.491438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.491413    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:47:46.594955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.594919    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-socket\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.594963    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-data\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.594982    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-credential-socket\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595010    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-kube-api-access-vfdjf\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-kube-api-access-vfdjf\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595042    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-token\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-token\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595090    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5079682a-1b91-4bbf-b05c-d521f51e499b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5079682a-1b91-4bbf-b05c-d521f51e499b-istiod-ca-cert\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595138    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-certs\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595162    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-podinfo\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595208    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-envoy\") pod \"5079682a-1b91-4bbf-b05c-d521f51e499b\" (UID: \"5079682a-1b91-4bbf-b05c-d521f51e499b\") "
Oct 30 20:47:46.595499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595248    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-socket" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:46.595499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595273    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-data" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:46.595499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595317    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-credential-socket" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:46.595754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595563    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-certs" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:46.595754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595579    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5079682a-1b91-4bbf-b05c-d521f51e499b-istiod-ca-cert" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:47:46.595754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595689    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.595754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595704    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.595754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595715    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5079682a-1b91-4bbf-b05c-d521f51e499b-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.595754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595729    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.595754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.595742    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.597483 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.597455    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-envoy" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:47:46.597596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.597509    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-podinfo" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:47:46.597780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.597761    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-token" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:46.597780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.597775    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-kube-api-access-vfdjf" pod "5079682a-1b91-4bbf-b05c-d521f51e499b" (UID: "5079682a-1b91-4bbf-b05c-d521f51e499b"). InnerVolumeSpecName "kube-api-access-vfdjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:47:46.609553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.609529    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 20:47:46.609666 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.609594    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 20:47:46.679455 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.679421    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:46.696850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.696814    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfdjf\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-kube-api-access-vfdjf\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.696850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.696848    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.696850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.696863    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.697120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.696877    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5079682a-1b91-4bbf-b05c-d521f51e499b-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:47:46.727279 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.727238    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r" event={"ID":"cbc65f10-d54b-4238-82ff-71b4a623b7ca","Type":"ContainerStarted","Data":"562ca65dbffa2a91f5b77ac0985dccf0103b7463024fad7a3868a204a383ce77"}
Oct 30 20:47:46.727799 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.727443    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:46.729143 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.729110    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" event={"ID":"c70bde44-f354-4a89-ac63-af5e6300f8fc","Type":"ContainerStarted","Data":"97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3"}
Oct 30 20:47:46.729371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.729339    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" event={"ID":"c70bde44-f354-4a89-ac63-af5e6300f8fc","Type":"ContainerStarted","Data":"fe4fb2bffb4b2dc844eefef70f08c68b762966eac6d85fc49d893485fa41960e"}
Oct 30 20:47:46.729506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.729379    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:46.730959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.730936    4852 generic.go:356] "Generic (PLEG): container finished" podID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerID="f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1" exitCode=0
Oct 30 20:47:46.731069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.730991    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"
Oct 30 20:47:46.731069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.731020    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" event={"ID":"5079682a-1b91-4bbf-b05c-d521f51e499b","Type":"ContainerDied","Data":"f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1"}
Oct 30 20:47:46.731069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.731061    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8" event={"ID":"5079682a-1b91-4bbf-b05c-d521f51e499b","Type":"ContainerDied","Data":"01513d61583e2c1d3e635839407b6765403e44c7536770222059f8bf1ac29c2c"}
Oct 30 20:47:46.731233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.731078    4852 scope.go:122] "RemoveContainer" containerID="f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1"
Oct 30 20:47:46.741591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.741573    4852 scope.go:122] "RemoveContainer" containerID="9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392"
Oct 30 20:47:46.750572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.750547    4852 scope.go:122] "RemoveContainer" containerID="f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1"
Oct 30 20:47:46.750871 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:46.750850    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1\": container with ID starting with f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1 not found: ID does not exist" containerID="f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1"
Oct 30 20:47:46.750926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.750879    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1"} err="failed to get container status \"f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1\": rpc error: code = NotFound desc = could not find container \"f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1\": container with ID starting with f22c1183b4fdb197f9e609590e558206399ce0dcccc5977ee57c59ac875780e1 not found: ID does not exist"
Oct 30 20:47:46.750926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.750900    4852 scope.go:122] "RemoveContainer" containerID="9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392"
Oct 30 20:47:46.751109 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:46.751093    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392\": container with ID starting with 9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392 not found: ID does not exist" containerID="9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392"
Oct 30 20:47:46.751151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.751112    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392"} err="failed to get container status \"9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392\": rpc error: code = NotFound desc = could not find container \"9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392\": container with ID starting with 9960e714223540c78a384bc150f01059c532343808b828911b9ed5a227c22392 not found: ID does not exist"
Oct 30 20:47:46.751728 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.751691    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r" podStartSLOduration=2.285111464 podStartE2EDuration="2.751678765s" totalImagesPullingTime="466.567301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:46.749270812 +0000 UTC m=+574.864004995" watchObservedRunningTime="2026-10-30 20:47:46.751678765 +0000 UTC m=+574.866412955"
Oct 30 20:47:46.770827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.770777    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" podStartSLOduration=1.770763107 podStartE2EDuration="1.770763107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:46.769087253 +0000 UTC m=+574.883821449" watchObservedRunningTime="2026-10-30 20:47:46.770763107 +0000 UTC m=+574.885497296"
Oct 30 20:47:46.786082 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.786051    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"]
Oct 30 20:47:46.790959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:46.790937    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-67598f1d-openshift-default-cfc6b86c7-pm6v8"]
Oct 30 20:47:47.382137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.382101    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:47.476478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.476435    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"]
Oct 30 20:47:47.476883 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.476869    4852 cpu_manager.go:414] "containerMap: removing container" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy"
Oct 30 20:47:47.476923 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.476884    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy"
Oct 30 20:47:47.476923 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.476919    4852 cpu_manager.go:414] "containerMap: removing container" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy"
Oct 30 20:47:47.476990 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.476928    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy"
Oct 30 20:47:47.477020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.476992    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy"
Oct 30 20:47:47.477020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.477008    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" containerName="istio-proxy"
Oct 30 20:47:47.481834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.481812    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.486259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.486229    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-8e4acb\""
Oct 30 20:47:47.488762 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.488737    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:47.489473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.489450    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"]
Oct 30 20:47:47.523539 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.523504    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"]
Oct 30 20:47:47.527268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.527248    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:47.543507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.543470    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-8e4acb-dockercfg-h68nn\""
Oct 30 20:47:47.544427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.544315    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"]
Oct 30 20:47:47.606851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.606813    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b5e064-066d-40f1-badb-15b3f5d9a641-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.607014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.606868    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnms7\" (UniqueName: \"kubernetes.io/projected/73b5e064-066d-40f1-badb-15b3f5d9a641-kube-api-access-rnms7\") pod \"payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.707982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.707887    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hlws\" (UniqueName: \"kubernetes.io/projected/e7b0562c-6755-48c3-a8df-c26a16ad4c22-kube-api-access-6hlws\") pod \"payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:47.708145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.708037    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b5e064-066d-40f1-badb-15b3f5d9a641-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.708145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.708078    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b0562c-6755-48c3-a8df-c26a16ad4c22-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:47.708145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.708104    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnms7\" (UniqueName: \"kubernetes.io/projected/73b5e064-066d-40f1-badb-15b3f5d9a641-kube-api-access-rnms7\") pod \"payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.708774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.708749    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b5e064-066d-40f1-badb-15b3f5d9a641-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.717900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.717870    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnms7\" (UniqueName: \"kubernetes.io/projected/73b5e064-066d-40f1-badb-15b3f5d9a641-kube-api-access-rnms7\") pod \"payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.799034 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.798992    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:47.809666 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.809642    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hlws\" (UniqueName: \"kubernetes.io/projected/e7b0562c-6755-48c3-a8df-c26a16ad4c22-kube-api-access-6hlws\") pod \"payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:47.809766 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.809743    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b0562c-6755-48c3-a8df-c26a16ad4c22-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:47.810500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.810481    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b0562c-6755-48c3-a8df-c26a16ad4c22-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:47.819270 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.819241    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hlws\" (UniqueName: \"kubernetes.io/projected/e7b0562c-6755-48c3-a8df-c26a16ad4c22-kube-api-access-6hlws\") pod \"payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:47.841707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:47.841675    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:48.004097 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.004016    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5079682a-1b91-4bbf-b05c-d521f51e499b" path="/var/lib/kubelet/pods/5079682a-1b91-4bbf-b05c-d521f51e499b/volumes"
Oct 30 20:47:48.080221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.080190    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"]
Oct 30 20:47:48.081866 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:48.081840    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b5e064_066d_40f1_badb_15b3f5d9a641.slice/crio-3b43c8410dc54dc974a1906cf38db6ac1ed85b78e15616436e58867b29100d0f WatchSource:0}: Error finding container 3b43c8410dc54dc974a1906cf38db6ac1ed85b78e15616436e58867b29100d0f: Status 404 returned error can't find the container with id 3b43c8410dc54dc974a1906cf38db6ac1ed85b78e15616436e58867b29100d0f
Oct 30 20:47:48.102254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.102224    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:48.139968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.139901    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"]
Oct 30 20:47:48.142299 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:48.142272    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b0562c_6755_48c3_a8df_c26a16ad4c22.slice/crio-03f03b236e17b30e5a4e93e2ffa346c64b34b10a7cb3279dd96893e4fe8ce6a6 WatchSource:0}: Error finding container 03f03b236e17b30e5a4e93e2ffa346c64b34b10a7cb3279dd96893e4fe8ce6a6: Status 404 returned error can't find the container with id 03f03b236e17b30e5a4e93e2ffa346c64b34b10a7cb3279dd96893e4fe8ce6a6
Oct 30 20:47:48.747187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.747146    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" event={"ID":"73b5e064-066d-40f1-badb-15b3f5d9a641","Type":"ContainerStarted","Data":"1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39"}
Oct 30 20:47:48.747432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.747233    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" event={"ID":"73b5e064-066d-40f1-badb-15b3f5d9a641","Type":"ContainerStarted","Data":"3b43c8410dc54dc974a1906cf38db6ac1ed85b78e15616436e58867b29100d0f"}
Oct 30 20:47:48.747432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.747365    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:47:48.748803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.748776    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" event={"ID":"e7b0562c-6755-48c3-a8df-c26a16ad4c22","Type":"ContainerStarted","Data":"2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338"}
Oct 30 20:47:48.748906 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.748821    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:48.748906 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.748841    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" event={"ID":"e7b0562c-6755-48c3-a8df-c26a16ad4c22","Type":"ContainerStarted","Data":"03f03b236e17b30e5a4e93e2ffa346c64b34b10a7cb3279dd96893e4fe8ce6a6"}
Oct 30 20:47:48.779809 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.779752    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" podStartSLOduration=1.779738008 podStartE2EDuration="1.779738008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:48.77687168 +0000 UTC m=+576.891605871" watchObservedRunningTime="2026-10-30 20:47:48.779738008 +0000 UTC m=+576.894472198"
Oct 30 20:47:48.793303 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:48.793064    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" podStartSLOduration=1.793047891 podStartE2EDuration="1.793047891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:48.792849133 +0000 UTC m=+576.907583336" watchObservedRunningTime="2026-10-30 20:47:48.793047891 +0000 UTC m=+576.907782084"
Oct 30 20:47:49.102696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.102651    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"]
Oct 30 20:47:49.107034 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.107011    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.110781 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.110758    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-bc102e-serving-cert\""
Oct 30 20:47:49.119035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.119004    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"]
Oct 30 20:47:49.141582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.141545    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:49.221971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.221937    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:49.224009 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.223976    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d5e04b-1973-4553-8026-555abdc93f9d-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.224163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.224063    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-maas-api-tls\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.224163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.224108    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-metrics-certs\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.224239 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.224182    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwrbq\" (UniqueName: \"kubernetes.io/projected/27d5e04b-1973-4553-8026-555abdc93f9d-kube-api-access-hwrbq\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.324979 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.324940    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d5e04b-1973-4553-8026-555abdc93f9d-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.325183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.325018    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-maas-api-tls\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.325183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.325037    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-metrics-certs\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.325183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.325059    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwrbq\" (UniqueName: \"kubernetes.io/projected/27d5e04b-1973-4553-8026-555abdc93f9d-kube-api-access-hwrbq\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.325685 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.325658    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d5e04b-1973-4553-8026-555abdc93f9d-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.327629 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.327605    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-metrics-certs\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.327754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.327731    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-maas-api-tls\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.334153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.334122    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwrbq\" (UniqueName: \"kubernetes.io/projected/27d5e04b-1973-4553-8026-555abdc93f9d-kube-api-access-hwrbq\") pod \"maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.422374 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.422274    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:49.585294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.585261    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:49.720471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.720382    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"]
Oct 30 20:47:49.722306 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:49.722280    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d5e04b_1973_4553_8026_555abdc93f9d.slice/crio-6daaab83ec3afc849e49e0532df5ad529b41a4ad126a06493c02720b639aa5a2 WatchSource:0}: Error finding container 6daaab83ec3afc849e49e0532df5ad529b41a4ad126a06493c02720b639aa5a2: Status 404 returned error can't find the container with id 6daaab83ec3afc849e49e0532df5ad529b41a4ad126a06493c02720b639aa5a2
Oct 30 20:47:49.785715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:49.785672    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h" event={"ID":"27d5e04b-1973-4553-8026-555abdc93f9d","Type":"ContainerStarted","Data":"6daaab83ec3afc849e49e0532df5ad529b41a4ad126a06493c02720b639aa5a2"}
Oct 30 20:47:50.082705 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:50.082666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:50.195568 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:50.195533    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:50.299931 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:50.299903    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:50.403157 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:50.403122    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:50.793744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:50.793708    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h" event={"ID":"27d5e04b-1973-4553-8026-555abdc93f9d","Type":"ContainerStarted","Data":"72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56"}
Oct 30 20:47:50.793931 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:50.793774    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:50.815252 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:50.815205    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h" podStartSLOduration=1.24774288 podStartE2EDuration="1.815191638s" totalImagesPullingTime="567.448758ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:50.81248165 +0000 UTC m=+578.927215841" watchObservedRunningTime="2026-10-30 20:47:50.815191638 +0000 UTC m=+578.929925828"
Oct 30 20:47:51.315758 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:51.315724    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:51.579803 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:47:51.579714    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:47:52.110969 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.110929    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"]
Oct 30 20:47:52.116516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.116487    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.120716 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.120685    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"]
Oct 30 20:47:52.122941 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.122912    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-f9a819\"/\"kube-root-ca.crt\""
Oct 30 20:47:52.123092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.123069    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-f9a819\"/\"openshift-service-ca.crt\""
Oct 30 20:47:52.123190 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.123101    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-f9a819\"/\"simulator-w3-f9a819-kserve-self-signed-certs\""
Oct 30 20:47:52.123290 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.123274    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-f9a819\"/\"default-dockercfg-mvzcp\""
Oct 30 20:47:52.187313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.187278    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:52.275021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.274972    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-dshm\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.275185 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.275068    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-tmp-dir\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.275185 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.275131    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-home\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.275185 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.275170    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75zdv\" (UniqueName: \"kubernetes.io/projected/66dd404d-7667-4c38-bff7-3dc844d263b7-kube-api-access-75zdv\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.275317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.275215    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66dd404d-7667-4c38-bff7-3dc844d263b7-tls-certs\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.275317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.275275    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-model-cache\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.374486 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.373895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:52.376744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.376720    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66dd404d-7667-4c38-bff7-3dc844d263b7-tls-certs\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.376862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.376766    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-model-cache\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.376862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.376815    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-dshm\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.376862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.376856    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-tmp-dir\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.376997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.376907    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-home\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.376997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.376939    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75zdv\" (UniqueName: \"kubernetes.io/projected/66dd404d-7667-4c38-bff7-3dc844d263b7-kube-api-access-75zdv\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.377687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.377662    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-model-cache\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.377782 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.377702    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-tmp-dir\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.377782 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.377765    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-home\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.379581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.379561    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-dshm\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.380029 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.380014    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66dd404d-7667-4c38-bff7-3dc844d263b7-tls-certs\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.387464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.387442    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75zdv\" (UniqueName: \"kubernetes.io/projected/66dd404d-7667-4c38-bff7-3dc844d263b7-kube-api-access-75zdv\") pod \"simulator-w3-f9a819-kserve-74897965c6-kr6ft\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") " pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.431656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.431618    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:52.497969 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.497917    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:52.707249 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.707164    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:52.746228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.746202    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 20:47:52.779459 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.779366    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"]
Oct 30 20:47:52.805350 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.805318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:52.809672 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:52.809641    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" event={"ID":"66dd404d-7667-4c38-bff7-3dc844d263b7","Type":"ContainerStarted","Data":"a612dcaba79af395ab3d6d5a372574c2cfad2b04bc3871c2305e31021d06bb2c"}
Oct 30 20:47:53.371410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:53.370939    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:53.486477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:53.482324    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:53.580820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:53.580786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:53.818294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:53.818256    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" event={"ID":"66dd404d-7667-4c38-bff7-3dc844d263b7","Type":"ContainerStarted","Data":"99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554"}
Oct 30 20:47:53.876360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:53.876299    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podStartSLOduration=1.876282684 podStartE2EDuration="1.876282684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:53.87469786 +0000 UTC m=+581.989432052" watchObservedRunningTime="2026-10-30 20:47:53.876282684 +0000 UTC m=+581.991016873"
Oct 30 20:47:54.431873 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:54.431841    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:47:54.555759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:54.555725    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:55.679395 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:55.679342    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:55.720538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:55.720508    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 20:47:55.720733 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:55.720588    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 20:47:56.695651 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:56.695606    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:47:56.696071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:56.695680    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:47:56.804904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:56.804871    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 20:47:56.880411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:56.872448    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:57.743506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.743420    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 20:47:57.824013 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.823979    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"]
Oct 30 20:47:57.827505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.827483    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.836676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.836656    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-53aa47\"/\"kube-root-ca.crt\""
Oct 30 20:47:57.837338 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.837044    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-53aa47\"/\"default-dockercfg-b57dx\""
Oct 30 20:47:57.837473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.837117    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-53aa47\"/\"simulator-w4-53aa47-kserve-self-signed-certs\""
Oct 30 20:47:57.838020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.837302    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-53aa47\"/\"openshift-service-ca.crt\""
Oct 30 20:47:57.840639 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.840617    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"]
Oct 30 20:47:57.849290 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.849252    4852 generic.go:356] "Generic (PLEG): container finished" podID="642feace-727f-49a7-8955-b895832d2540" containerID="836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447" exitCode=0
Oct 30 20:47:57.849439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.849292    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" event={"ID":"642feace-727f-49a7-8955-b895832d2540","Type":"ContainerDied","Data":"836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447"}
Oct 30 20:47:57.858182 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.849688    4852 scope.go:122] "RemoveContainer" containerID="836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447"
Oct 30 20:47:57.878378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.878286    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:57.878378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.878397    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:57.878754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.878471    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:47:57.879144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.879052    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-model-cache\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.879144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.879130    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tls-certs\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.880668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.879251    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-dshm\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.880668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.879320    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25m4h\" (UniqueName: \"kubernetes.io/projected/870f82ea-35a6-4f67-a2ce-d1c2983fd410-kube-api-access-25m4h\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.880668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.879372    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tmp-dir\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.880668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.879414    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-home\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.880668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.880622    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:57.978954 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.977272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981267    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-model-cache\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981329    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tls-certs\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981371    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-dshm\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981455    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25m4h\" (UniqueName: \"kubernetes.io/projected/870f82ea-35a6-4f67-a2ce-d1c2983fd410-kube-api-access-25m4h\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981493    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tmp-dir\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981510    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-home\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981817    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-home\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.981984    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-model-cache\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.982433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.982357    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tmp-dir\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.984824 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.984801    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-dshm\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.985108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.985086    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tls-certs\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:57.991239 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:57.991218    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25m4h\" (UniqueName: \"kubernetes.io/projected/870f82ea-35a6-4f67-a2ce-d1c2983fd410-kube-api-access-25m4h\") pod \"simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") " pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:58.142370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:58.142283    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:58.461708 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:47:58.461657    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod870f82ea_35a6_4f67_a2ce_d1c2983fd410.slice/crio-94d5b2d4fb3f5d0a151865c697d59f95f050f7219a7f65fc93b4dba16e306d3f WatchSource:0}: Error finding container 94d5b2d4fb3f5d0a151865c697d59f95f050f7219a7f65fc93b4dba16e306d3f: Status 404 returned error can't find the container with id 94d5b2d4fb3f5d0a151865c697d59f95f050f7219a7f65fc93b4dba16e306d3f
Oct 30 20:47:58.478412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:58.462493    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"]
Oct 30 20:47:58.878346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:58.878308    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" event={"ID":"642feace-727f-49a7-8955-b895832d2540","Type":"ContainerStarted","Data":"2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687"}
Oct 30 20:47:58.882147 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:58.882111    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" event={"ID":"870f82ea-35a6-4f67-a2ce-d1c2983fd410","Type":"ContainerStarted","Data":"a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1"}
Oct 30 20:47:58.882430 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:58.882404    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" event={"ID":"870f82ea-35a6-4f67-a2ce-d1c2983fd410","Type":"ContainerStarted","Data":"94d5b2d4fb3f5d0a151865c697d59f95f050f7219a7f65fc93b4dba16e306d3f"}
Oct 30 20:47:58.939682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:58.939628    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podStartSLOduration=1.9396140320000002 podStartE2EDuration="1.939614032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:47:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:47:58.936046684 +0000 UTC m=+587.050780873" watchObservedRunningTime="2026-10-30 20:47:58.939614032 +0000 UTC m=+587.054348222"
Oct 30 20:47:58.941443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:58.941409    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:59.042934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.042879    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:59.080412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.076859    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:47:59.080412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.076981    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 20:47:59.080412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.077028    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 20:47:59.142649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.142552    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:47:59.171423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.169316    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:47:59.821859 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.821823    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 20:47:59.822041 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:47:59.821936    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 20:48:00.083467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:00.082525    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 20:48:00.083467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:00.082595    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 20:48:00.114414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:00.107486    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:00.928755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:00.928692    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:01.076915 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:01.076875    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 20:48:01.077218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:01.077184    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 20:48:01.747934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:01.747891    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:02.054963 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.054910    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:02.077891 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.077850    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 20:48:02.078186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.078154    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 20:48:02.432628 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.432532    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:48:02.445220 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.445192    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:48:02.691310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.691200    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:02.806179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.806129    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:02.901319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.901291    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:02.941821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:02.941722    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:48:03.076934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:03.076893    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 20:48:03.077121 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:03.076964    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 20:48:03.126837 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:03.126807    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:03.964522 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:03.964487    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:04.063753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:04.063718    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:04.077355 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:04.077317    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 20:48:04.077561 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:04.077370    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 20:48:04.612717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:04.612682    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:04.765284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:04.765252    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:04.864579 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:04.864496    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:05.080641 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.080604    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:48:05.081050 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.081031    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:48:05.081933 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.081913    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 20:48:05.102373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.102345    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:05.956145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.956114    4852 generic.go:356] "Generic (PLEG): container finished" podID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerID="2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1" exitCode=0
Oct 30 20:48:05.956373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.956184    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" event={"ID":"c925dcc8-57c5-47aa-9dd4-a011fadf74da","Type":"ContainerDied","Data":"2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1"}
Oct 30 20:48:05.956633 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.956616    4852 scope.go:122] "RemoveContainer" containerID="2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1"
Oct 30 20:48:05.957810 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.957784    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:48:05.957916 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.957866    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:48:05.957916 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:05.957911    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:48:06.022545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:06.022511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:06.978988 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:06.978943    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" event={"ID":"c925dcc8-57c5-47aa-9dd4-a011fadf74da","Type":"ContainerStarted","Data":"af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982"}
Oct 30 20:48:07.075893 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:07.075847    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:07.901339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:07.901292    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:48:07.901667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:07.901629    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 20:48:07.901822 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:07.901684    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 20:48:08.123838 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:08.123778    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:08.142682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:08.142645    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:48:08.188633 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:08.188550    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:48:08.902065 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:08.902032    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 20:48:08.902261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:08.902086    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 20:48:09.007560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:09.007534    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:48:09.031152 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:09.031120    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:09.735497 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:09.735455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:09.901116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:09.901077    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 20:48:09.901301 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:09.901162    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 20:48:09.971409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:09.970847    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:10.632744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:10.632709    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:10.901538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:10.901457    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 20:48:10.901538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:10.901513    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 20:48:11.301359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:11.301326    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:11.401071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:11.401041    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:11.661177 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:11.660989    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:11.766945 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:11.766858    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:11.901710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:11.901671    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 20:48:11.902104 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:11.901751    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 20:48:11.951125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:11.951051    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:12.068261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:12.068213    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:12.203378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:12.203277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:12.905523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:12.905494    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:48:12.905920 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:12.905883    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:48:12.906676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:12.906638    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 20:48:13.181955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:13.181865    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:13.282128 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:13.282073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:13.385544 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:13.385513    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:14.039581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:14.039548    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:14.929890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:14.929841    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:15.841731 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:15.841694    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:16.841080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:16.841040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:18.156625 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.156593    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:18.721093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.721049    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"]
Oct 30 20:48:18.728044 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.728011    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.730375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.730344    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"]
Oct 30 20:48:18.733449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.733414    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-8e4acb\"/\"kube-root-ca.crt\""
Oct 30 20:48:18.733449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.733426    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-8e4acb\"/\"default-dockercfg-5f97g\""
Oct 30 20:48:18.733449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.733426    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-8e4acb\"/\"simulator-w2-8e4acb-kserve-self-signed-certs\""
Oct 30 20:48:18.733792 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.733769    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-8e4acb\"/\"openshift-service-ca.crt\""
Oct 30 20:48:18.764149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.764115    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-tmp-dir\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.764149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.764153    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-dshm\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.764363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.764173    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-home\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.764363 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.764337    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-model-cache\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.764476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.764378    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0540acbd-331d-4277-b186-acf474a45624-tls-certs\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.764476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.764415    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2pt4\" (UniqueName: \"kubernetes.io/projected/0540acbd-331d-4277-b186-acf474a45624-kube-api-access-t2pt4\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.774046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.774017    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:18.865915 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.865881    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-model-cache\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.865934    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0540acbd-331d-4277-b186-acf474a45624-tls-certs\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.865954    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2pt4\" (UniqueName: \"kubernetes.io/projected/0540acbd-331d-4277-b186-acf474a45624-kube-api-access-t2pt4\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.866000    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-tmp-dir\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.866019    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-dshm\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866267 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.866126    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-home\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866338 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.866311    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-model-cache\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866406 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.866366    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-tmp-dir\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.866523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.866503    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-home\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.868428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.868402    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-dshm\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.868634 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.868609    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0540acbd-331d-4277-b186-acf474a45624-tls-certs\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.874932 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.874854    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2pt4\" (UniqueName: \"kubernetes.io/projected/0540acbd-331d-4277-b186-acf474a45624-kube-api-access-t2pt4\") pod \"simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") " pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:18.972514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:18.972443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:19.040981 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:19.040948    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:19.324717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:19.324687    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"]
Oct 30 20:48:19.329790 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:48:19.326194    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0540acbd_331d_4277_b186_acf474a45624.slice/crio-3019b8aced0b55a25cb28385c8386fbd94901e3e09031c0b94d342520c5bdfb9 WatchSource:0}: Error finding container 3019b8aced0b55a25cb28385c8386fbd94901e3e09031c0b94d342520c5bdfb9: Status 404 returned error can't find the container with id 3019b8aced0b55a25cb28385c8386fbd94901e3e09031c0b94d342520c5bdfb9
Oct 30 20:48:19.862339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:19.862301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:20.070784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:20.070739    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" event={"ID":"0540acbd-331d-4277-b186-acf474a45624","Type":"ContainerStarted","Data":"a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c"}
Oct 30 20:48:20.070784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:20.070789    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" event={"ID":"0540acbd-331d-4277-b186-acf474a45624","Type":"ContainerStarted","Data":"3019b8aced0b55a25cb28385c8386fbd94901e3e09031c0b94d342520c5bdfb9"}
Oct 30 20:48:20.092784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:20.092734    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podStartSLOduration=2.092719726 podStartE2EDuration="2.092719726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:48:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:48:20.089416502 +0000 UTC m=+608.204150691" watchObservedRunningTime="2026-10-30 20:48:20.092719726 +0000 UTC m=+608.207453909"
Oct 30 20:48:20.735644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:20.735609    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:21.041491 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:21.041456    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:21.559527 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:48:21.559492    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:48:21.716284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:21.716236    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:22.629863 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:22.629827    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:23.578163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:23.578133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:24.323623 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.323587    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:24.398028 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.397997    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"]
Oct 30 20:48:24.406245 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.406218    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.420733 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.420709    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-317a33\"/\"default-dockercfg-pwtr5\""
Oct 30 20:48:24.420977 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.420950    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-317a33\"/\"premium-w1-317a33-kserve-self-signed-certs\""
Oct 30 20:48:24.423167 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.423145    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-317a33\"/\"openshift-service-ca.crt\""
Oct 30 20:48:24.423298 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.423281    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-317a33\"/\"kube-root-ca.crt\""
Oct 30 20:48:24.425789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.425767    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"]
Oct 30 20:48:24.524801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.524768    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-tmp-dir\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.524991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.524820    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775b7bb3-76c8-41fd-876b-f3646c25b77b-tls-certs\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.524991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.524907    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-home\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.525086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.525031    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4zxs\" (UniqueName: \"kubernetes.io/projected/775b7bb3-76c8-41fd-876b-f3646c25b77b-kube-api-access-h4zxs\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.525086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.525066    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-dshm\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.525086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.525082    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-model-cache\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626318    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-tmp-dir\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626367    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775b7bb3-76c8-41fd-876b-f3646c25b77b-tls-certs\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626411    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-home\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626471    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4zxs\" (UniqueName: \"kubernetes.io/projected/775b7bb3-76c8-41fd-876b-f3646c25b77b-kube-api-access-h4zxs\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626512    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-dshm\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626526    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-model-cache\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626893 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626835    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-tmp-dir\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626893 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626879    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-home\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.626992 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.626964    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-model-cache\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.628712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.628666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:24.628934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.628916    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-dshm\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.629061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.629045    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775b7bb3-76c8-41fd-876b-f3646c25b77b-tls-certs\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.641325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.641302    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4zxs\" (UniqueName: \"kubernetes.io/projected/775b7bb3-76c8-41fd-876b-f3646c25b77b-kube-api-access-h4zxs\") pod \"premium-w1-317a33-kserve-5b7f75fd95-ms7x4\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") " pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:24.721336 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:24.721304    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:25.004004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:25.003976    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"]
Oct 30 20:48:25.005322 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:48:25.005291    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod775b7bb3_76c8_41fd_876b_f3646c25b77b.slice/crio-fb6f9d09f150bd534c3b74077fd32fd5ad2caa7e090e1dba57c9771d94173f91 WatchSource:0}: Error finding container fb6f9d09f150bd534c3b74077fd32fd5ad2caa7e090e1dba57c9771d94173f91: Status 404 returned error can't find the container with id fb6f9d09f150bd534c3b74077fd32fd5ad2caa7e090e1dba57c9771d94173f91
Oct 30 20:48:25.113197 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:25.113161    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" event={"ID":"775b7bb3-76c8-41fd-876b-f3646c25b77b","Type":"ContainerStarted","Data":"4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd"}
Oct 30 20:48:25.113380 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:25.113211    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" event={"ID":"775b7bb3-76c8-41fd-876b-f3646c25b77b","Type":"ContainerStarted","Data":"fb6f9d09f150bd534c3b74077fd32fd5ad2caa7e090e1dba57c9771d94173f91"}
Oct 30 20:48:25.131812 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:25.131751    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podStartSLOduration=1.131736727 podStartE2EDuration="1.131736727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:48:25.130090644 +0000 UTC m=+613.244824833" watchObservedRunningTime="2026-10-30 20:48:25.131736727 +0000 UTC m=+613.246470917"
Oct 30 20:48:25.476073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:25.476039    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:25.722224 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:25.722194    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:26.281792 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:26.281761    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:27.145744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:27.145692    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:28.456928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:28.456899    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:29.041312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:29.041276    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:29.053853 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:29.053827    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:29.150691 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:29.150648    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:48:29.344403 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:29.344299    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:30.143797 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:30.143760    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:30.303378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:30.303314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:31.037452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:31.037414    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:32.123042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:32.123009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:32.846711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:32.846674    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:33.693022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:33.692983    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:34.721861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:34.721829    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:34.735095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:34.735066    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:34.881262 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:34.881218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:35.188499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:35.188470    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:48:35.844006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:35.843971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:36.580458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:36.580417    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:36.877141 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:36.877058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:37.860365 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:37.860329    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:38.711294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:38.711249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:39.766023 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:39.765990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:40.753581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:40.753532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:41.225511 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:41.225474    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:41.959712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:41.959679    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:43.032872 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:43.032839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:43.850015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:43.849971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:44.636826 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:44.636792    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:45.441263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:45.441227    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:46.255114 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:46.255074    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:47.315886 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:47.315839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:48.453033 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:48.452997    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:49.252759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:49.252728    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:50.018608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:50.018576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:50.843809 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:50.843770    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:51.557547 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:48:51.557514    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:48:51.758812 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:51.758777    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:52.677325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:52.677293    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:53.562101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:53.562069    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:54.393188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:54.393152    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:55.449353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:55.449318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:56.574898 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:56.574863    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:57.779829 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:57.779793    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:58.756855 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:58.756818    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:59.248150 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.248068    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"]
Oct 30 20:48:59.252919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.252893    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.271080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.271047    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"]
Oct 30 20:48:59.293424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293360    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-credential-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293422    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-envoy\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293512    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6dm6\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-kube-api-access-m6dm6\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293551    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-istio-token\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293792 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293709    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293849 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293801    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4214ba5f-63d1-4448-abad-833ebbf18022-istiod-ca-cert\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293863    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-data\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293962 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293917    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-certs\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.293962 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.293944    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4214ba5f-63d1-4448-abad-833ebbf18022-istio-podinfo\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.338712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.338680    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-53aa47-openshift-default-dockercfg-flpcg\""
Oct 30 20:48:59.395502 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395463    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.395709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395530    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4214ba5f-63d1-4448-abad-833ebbf18022-istiod-ca-cert\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.395709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395567    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-data\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.395709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395599    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-certs\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.395709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395620    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4214ba5f-63d1-4448-abad-833ebbf18022-istio-podinfo\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.395709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395673    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-credential-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.395709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395698    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-envoy\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.396022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395748    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6dm6\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-kube-api-access-m6dm6\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.396022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395778    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-istio-token\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.396022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.395949    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.396186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.396047    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-certs\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.396186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.396110    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-credential-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.396294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.396220    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-data\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.396377 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.396360    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4214ba5f-63d1-4448-abad-833ebbf18022-istiod-ca-cert\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.398068 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.398044    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-envoy\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.398170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.398160    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4214ba5f-63d1-4448-abad-833ebbf18022-istio-podinfo\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.405088 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.405058    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-istio-token\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.405953 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.405926    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6dm6\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-kube-api-access-m6dm6\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.568227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.568184    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:48:59.775081 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.775037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:48:59.860512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.860482    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"]
Oct 30 20:48:59.862376 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:48:59.862350    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4214ba5f_63d1_4448_abad_833ebbf18022.slice/crio-9d19c877a3791e4b7d4a39c22ec37a8bbedf8baf94b7bb5f6212e7741593555d WatchSource:0}: Error finding container 9d19c877a3791e4b7d4a39c22ec37a8bbedf8baf94b7bb5f6212e7741593555d: Status 404 returned error can't find the container with id 9d19c877a3791e4b7d4a39c22ec37a8bbedf8baf94b7bb5f6212e7741593555d
Oct 30 20:48:59.864838 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.864805    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:48:59.864919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.864872    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:48:59.864919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:48:59.864898    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:00.333741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:00.333689    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" event={"ID":"4214ba5f-63d1-4448-abad-833ebbf18022","Type":"ContainerStarted","Data":"bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c"}
Oct 30 20:49:00.334118 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:00.333777    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" event={"ID":"4214ba5f-63d1-4448-abad-833ebbf18022","Type":"ContainerStarted","Data":"9d19c877a3791e4b7d4a39c22ec37a8bbedf8baf94b7bb5f6212e7741593555d"}
Oct 30 20:49:00.373411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:00.371812    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podStartSLOduration=1.371792667 podStartE2EDuration="1.371792667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:00.369087677 +0000 UTC m=+648.483821868" watchObservedRunningTime="2026-10-30 20:49:00.371792667 +0000 UTC m=+648.486526856"
Oct 30 20:49:00.570422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:00.569736    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:49:00.574436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:00.573505    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 20:49:00.574436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:00.573566    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 20:49:00.646137 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:00.646048    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:01.532684 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.532648    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:01.569787 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.569749    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 20:49:01.570129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.570094    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 20:49:01.590504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.590472    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"]
Oct 30 20:49:01.594333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.594309    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.598834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.598815    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-f9a819-openshift-default-dockercfg-925js\""
Oct 30 20:49:01.610712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.610674    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"]
Oct 30 20:49:01.625924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.625881    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx2d2\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-kube-api-access-wx2d2\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.625938    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.626036    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.626084    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83b0578-9da4-47b6-b952-16cb884318ef-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626281 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.626106    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83b0578-9da4-47b6-b952-16cb884318ef-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626338 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.626283    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.626427    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626595 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.626578    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.626665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.626612    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.727982    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728028    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83b0578-9da4-47b6-b952-16cb884318ef-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728056    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83b0578-9da4-47b6-b952-16cb884318ef-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728206    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728264    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728380 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728342    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728378    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728429    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx2d2\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-kube-api-access-wx2d2\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728500    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728685 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728662    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.728773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.728754    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83b0578-9da4-47b6-b952-16cb884318ef-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.729068 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.729043    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.729161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.729095    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.729161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.729127    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.730587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.730559    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.730740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.730721    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83b0578-9da4-47b6-b952-16cb884318ef-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.736723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.736701    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.737019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.736995    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx2d2\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-kube-api-access-wx2d2\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:01.921248 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:01.921169    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:02.233963 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.233939    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"]
Oct 30 20:49:02.235640 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:02.235613    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83b0578_9da4_47b6_b952_16cb884318ef.slice/crio-6f55a9724150e47daeddebdd09b8b25594c0568fd9f3916e39418702faf5f3da WatchSource:0}: Error finding container 6f55a9724150e47daeddebdd09b8b25594c0568fd9f3916e39418702faf5f3da: Status 404 returned error can't find the container with id 6f55a9724150e47daeddebdd09b8b25594c0568fd9f3916e39418702faf5f3da
Oct 30 20:49:02.237927 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.237895    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:02.238039 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.237959    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:02.238039 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.237986    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:02.348134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.348102    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" event={"ID":"b83b0578-9da4-47b6-b952-16cb884318ef","Type":"ContainerStarted","Data":"7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117"}
Oct 30 20:49:02.348250 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.348163    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" event={"ID":"b83b0578-9da4-47b6-b952-16cb884318ef","Type":"ContainerStarted","Data":"6f55a9724150e47daeddebdd09b8b25594c0568fd9f3916e39418702faf5f3da"}
Oct 30 20:49:02.374417 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.374363    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:02.403430 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.401429    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podStartSLOduration=1.401404931 podStartE2EDuration="1.401404931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:02.393945919 +0000 UTC m=+650.508680110" watchObservedRunningTime="2026-10-30 20:49:02.401404931 +0000 UTC m=+650.516139123"
Oct 30 20:49:02.571414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.568874    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 20:49:02.571414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.568942    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 20:49:02.922307 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:02.922274    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:03.183241 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:03.183147    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:03.298237 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:03.298171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:03.569409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:03.568835    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 20:49:03.569409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:03.568902    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 20:49:03.923371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:03.923281    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:49:03.923371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:03.923353    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 20:49:04.037603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:04.037520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:04.572413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:04.570871    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 20:49:04.572413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:04.570937    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 20:49:04.926467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:04.923861    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:49:04.926467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:04.923927    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 20:49:04.948346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:04.948276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:05.568942 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:05.568905    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 20:49:05.569674 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:05.569641    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 20:49:05.922580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:05.922489    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:49:05.922580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:05.922553    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 20:49:06.044608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:06.044575    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:06.766748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:06.766716    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:49:06.768282 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:06.768261    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:49:06.778661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:06.778631    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:49:06.921894 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:06.921861    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:49:06.922063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:06.921918    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 20:49:06.971315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:06.971272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:07.921994 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:07.921958    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:49:07.922407 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:07.922015    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 20:49:07.975993 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:07.975948    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:08.741572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:08.741522    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:08.921751 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:08.921720    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 20:49:08.921927 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:08.921781    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 20:49:09.632234 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:09.632197    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:09.925739 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:09.925654    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:09.926075 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:09.926051    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:09.926693 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:09.926669    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 20:49:10.492702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.492667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:10.885214 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.885177    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"]
Oct 30 20:49:10.890112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.890093    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.903274 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.903256    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-bc102e-openshift-default-dockercfg-wm7tq\""
Oct 30 20:49:10.923489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.923462    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"]
Oct 30 20:49:10.967535 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967500    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.967535 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967538    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.967750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967556    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5z98\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-kube-api-access-l5z98\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.967750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967632    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a626ea-7131-4ae2-a793-c566838aec17-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.967750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967682    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.967750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967705    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a626ea-7131-4ae2-a793-c566838aec17-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.967922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967769    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.967922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967792    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:10.968023 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:10.967921    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.068964    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069009    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a626ea-7131-4ae2-a793-c566838aec17-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069034    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069058    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069117    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069175    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069202    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069225    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5z98\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-kube-api-access-l5z98\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069298    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a626ea-7131-4ae2-a793-c566838aec17-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069539    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.069871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069845    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a626ea-7131-4ae2-a793-c566838aec17-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.070015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.069970    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.070076 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.070057    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.070136 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.070092    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.071498 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.071475    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.071722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.071706    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a626ea-7131-4ae2-a793-c566838aec17-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.081613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.081586    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5z98\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-kube-api-access-l5z98\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.082620 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.082601    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.203371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.203290    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:11.490036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.490002    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"]
Oct 30 20:49:11.490982 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:11.490946    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a626ea_7131_4ae2_a793_c566838aec17.slice/crio-ec3f1388a699b076e63963e1b3a9e0a66246fe465b782c41b3fc6c95149af703 WatchSource:0}: Error finding container ec3f1388a699b076e63963e1b3a9e0a66246fe465b782c41b3fc6c95149af703: Status 404 returned error can't find the container with id ec3f1388a699b076e63963e1b3a9e0a66246fe465b782c41b3fc6c95149af703
Oct 30 20:49:11.493516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.493481    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:11.493588 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.493553    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:11.493588 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.493581    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:11.654089 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:11.654034    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:12.136019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.135990    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"]
Oct 30 20:49:12.139984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.139965    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.145530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.145498    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:49:12.146544 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.146521    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:49:12.159422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.149866    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"]
Oct 30 20:49:12.282133 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.282088    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prm69\" (UniqueName: \"kubernetes.io/projected/870d5026-8f4b-404f-9b4e-aa5164f35940-kube-api-access-prm69\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.282133 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.282139    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.282356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.282314    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.383879 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.383839    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.384080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.383917    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prm69\" (UniqueName: \"kubernetes.io/projected/870d5026-8f4b-404f-9b4e-aa5164f35940-kube-api-access-prm69\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.384080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.383940    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.384661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.384636    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.386286 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.386228    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.393675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.393648    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prm69\" (UniqueName: \"kubernetes.io/projected/870d5026-8f4b-404f-9b4e-aa5164f35940-kube-api-access-prm69\") pod \"maas-api-revoke-sub-3df8e679-p5wdd\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.474494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.472638    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:12.480517 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.480481    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" event={"ID":"75a626ea-7131-4ae2-a793-c566838aec17","Type":"ContainerStarted","Data":"21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d"}
Oct 30 20:49:12.480665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.480543    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" event={"ID":"75a626ea-7131-4ae2-a793-c566838aec17","Type":"ContainerStarted","Data":"ec3f1388a699b076e63963e1b3a9e0a66246fe465b782c41b3fc6c95149af703"}
Oct 30 20:49:12.502615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.502566    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podStartSLOduration=2.502551098 podStartE2EDuration="2.502551098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:12.501464812 +0000 UTC m=+660.616199003" watchObservedRunningTime="2026-10-30 20:49:12.502551098 +0000 UTC m=+660.617285289"
Oct 30 20:49:12.535330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.535283    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:12.778734 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:12.778696    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"]
Oct 30 20:49:12.782645 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:12.782615    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870d5026_8f4b_404f_9b4e_aa5164f35940.slice/crio-081f7d68b0aa8c3bf425533eff805c32806e57ee428431d9397af4f37534b895 WatchSource:0}: Error finding container 081f7d68b0aa8c3bf425533eff805c32806e57ee428431d9397af4f37534b895: Status 404 returned error can't find the container with id 081f7d68b0aa8c3bf425533eff805c32806e57ee428431d9397af4f37534b895
Oct 30 20:49:13.204239 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:13.204201    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:13.205706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:13.205681    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:49:13.205846 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:13.205726    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 20:49:13.324754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:13.324709    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:13.489129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:13.489040    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd" event={"ID":"870d5026-8f4b-404f-9b4e-aa5164f35940","Type":"ContainerStarted","Data":"78db8d6dce4cdcd77d8791e0d733592e0d2e59706644943cddbc0099097b9207"}
Oct 30 20:49:13.489129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:13.489099    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd" event={"ID":"870d5026-8f4b-404f-9b4e-aa5164f35940","Type":"ContainerStarted","Data":"081f7d68b0aa8c3bf425533eff805c32806e57ee428431d9397af4f37534b895"}
Oct 30 20:49:13.508156 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:13.508102    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd" podStartSLOduration=1.508088398 podStartE2EDuration="1.508088398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:13.506412202 +0000 UTC m=+661.621146382" watchObservedRunningTime="2026-10-30 20:49:13.508088398 +0000 UTC m=+661.622822587"
Oct 30 20:49:14.204276 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:14.204188    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:49:14.204276 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:14.204245    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 20:49:14.328555 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:14.328515    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:14.496732 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:14.496633    4852 generic.go:356] "Generic (PLEG): container finished" podID="870d5026-8f4b-404f-9b4e-aa5164f35940" containerID="78db8d6dce4cdcd77d8791e0d733592e0d2e59706644943cddbc0099097b9207" exitCode=0
Oct 30 20:49:14.496910 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:14.496733    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd" event={"ID":"870d5026-8f4b-404f-9b4e-aa5164f35940","Type":"ContainerDied","Data":"78db8d6dce4cdcd77d8791e0d733592e0d2e59706644943cddbc0099097b9207"}
Oct 30 20:49:15.024179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.024125    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:15.203747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.203710    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:49:15.203925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.203765    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 20:49:15.675779 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.675745    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:15.828768 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.828740    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-xks-ca\") pod \"870d5026-8f4b-404f-9b4e-aa5164f35940\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") "
Oct 30 20:49:15.828973 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.828789    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-openshift-ca\") pod \"870d5026-8f4b-404f-9b4e-aa5164f35940\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") "
Oct 30 20:49:15.828973 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.828813    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/870d5026-8f4b-404f-9b4e-aa5164f35940-kube-api-access-prm69\" (UniqueName: \"kubernetes.io/projected/870d5026-8f4b-404f-9b4e-aa5164f35940-kube-api-access-prm69\") pod \"870d5026-8f4b-404f-9b4e-aa5164f35940\" (UID: \"870d5026-8f4b-404f-9b4e-aa5164f35940\") "
Oct 30 20:49:15.829268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.829242    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-openshift-ca" pod "870d5026-8f4b-404f-9b4e-aa5164f35940" (UID: "870d5026-8f4b-404f-9b4e-aa5164f35940"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:49:15.830960 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.830936    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-xks-ca" pod "870d5026-8f4b-404f-9b4e-aa5164f35940" (UID: "870d5026-8f4b-404f-9b4e-aa5164f35940"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:49:15.831053 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.830960    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870d5026-8f4b-404f-9b4e-aa5164f35940-kube-api-access-prm69" pod "870d5026-8f4b-404f-9b4e-aa5164f35940" (UID: "870d5026-8f4b-404f-9b4e-aa5164f35940"). InnerVolumeSpecName "kube-api-access-prm69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:49:15.885818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.885755    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:15.929696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.929659    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:49:15.929696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.929693    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/870d5026-8f4b-404f-9b4e-aa5164f35940-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:49:15.929900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:15.929712    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prm69\" (UniqueName: \"kubernetes.io/projected/870d5026-8f4b-404f-9b4e-aa5164f35940-kube-api-access-prm69\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:49:16.208970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:16.208873    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:49:16.208970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:16.208933    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 20:49:16.509939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:16.509905    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"
Oct 30 20:49:16.509939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:16.509928    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd" event={"ID":"870d5026-8f4b-404f-9b4e-aa5164f35940","Type":"ContainerDied","Data":"081f7d68b0aa8c3bf425533eff805c32806e57ee428431d9397af4f37534b895"}
Oct 30 20:49:16.510158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:16.509976    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="081f7d68b0aa8c3bf425533eff805c32806e57ee428431d9397af4f37534b895"
Oct 30 20:49:16.882495 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:16.873994    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:17.204640 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:17.204559    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 20:49:17.204640 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:17.204615    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 20:49:17.738297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:17.738254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:17.838218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:17.838185    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:18.208457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:18.208421    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:18.208932 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:18.208841    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:18.209565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:18.209546    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 20:49:18.568848 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:18.568810    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:19.346821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:19.346791    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:20.289545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:20.289509    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:21.135671 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:21.135628    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:21.560811 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:49:21.560770    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:49:22.132752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:22.132720    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:23.137206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:23.137171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:24.045868 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:24.045830    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:24.954193 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:24.954156    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:25.497780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.497742    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"]
Oct 30 20:49:25.498267 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.498252    4852 cpu_manager.go:414] "containerMap: removing container" podUID="870d5026-8f4b-404f-9b4e-aa5164f35940" containerName="revoke-keys"
Oct 30 20:49:25.498311 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.498268    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="870d5026-8f4b-404f-9b4e-aa5164f35940" containerName="revoke-keys"
Oct 30 20:49:25.498360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.498351    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="870d5026-8f4b-404f-9b4e-aa5164f35940" containerName="revoke-keys"
Oct 30 20:49:25.502009 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.501993    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.517204 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.517168    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:49:25.517371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.517348    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:49:25.521151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.521121    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"]
Oct 30 20:49:25.648296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.648261    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwtxw\" (UniqueName: \"kubernetes.io/projected/2415084f-ae27-4510-83e0-5c84adca4ad6-kube-api-access-bwtxw\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.648296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.648305    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.648592 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.648437    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.662297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.662253    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:25.749839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.749743    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.750002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.749918    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwtxw\" (UniqueName: \"kubernetes.io/projected/2415084f-ae27-4510-83e0-5c84adca4ad6-kube-api-access-bwtxw\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.750002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.749956    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.750625 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.750600    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.752152 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.752128    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.759057 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.759033    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwtxw\" (UniqueName: \"kubernetes.io/projected/2415084f-ae27-4510-83e0-5c84adca4ad6-kube-api-access-bwtxw\") pod \"maas-api-revoke-sub-b91eb345-nhc46\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:25.764527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.764460    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:25.815763 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:25.815734    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:26.082888 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:26.082856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:26.168285 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:26.168255    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"]
Oct 30 20:49:26.169718 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:26.169689    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2415084f_ae27_4510_83e0_5c84adca4ad6.slice/crio-6d37b7e14702e25f179582107096957803a87b91cd6f1851c5f2a72d5307bd80 WatchSource:0}: Error finding container 6d37b7e14702e25f179582107096957803a87b91cd6f1851c5f2a72d5307bd80: Status 404 returned error can't find the container with id 6d37b7e14702e25f179582107096957803a87b91cd6f1851c5f2a72d5307bd80
Oct 30 20:49:26.251481 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:26.251452    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:26.579047 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:26.579007    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46" event={"ID":"2415084f-ae27-4510-83e0-5c84adca4ad6","Type":"ContainerStarted","Data":"6332071a2064e033a77a420baebf45e0eedb3dd84a5c194c738c3fa1de734fba"}
Oct 30 20:49:26.579233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:26.579066    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46" event={"ID":"2415084f-ae27-4510-83e0-5c84adca4ad6","Type":"ContainerStarted","Data":"6d37b7e14702e25f179582107096957803a87b91cd6f1851c5f2a72d5307bd80"}
Oct 30 20:49:26.600892 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:26.600835    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46" podStartSLOduration=1.600819761 podStartE2EDuration="1.600819761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:26.597528169 +0000 UTC m=+674.712262359" watchObservedRunningTime="2026-10-30 20:49:26.600819761 +0000 UTC m=+674.715553952"
Oct 30 20:49:26.955762 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:26.955720    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:27.591378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:27.591333    4852 generic.go:356] "Generic (PLEG): container finished" podID="2415084f-ae27-4510-83e0-5c84adca4ad6" containerID="6332071a2064e033a77a420baebf45e0eedb3dd84a5c194c738c3fa1de734fba" exitCode=0
Oct 30 20:49:27.593226 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:27.592889    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46" event={"ID":"2415084f-ae27-4510-83e0-5c84adca4ad6","Type":"ContainerDied","Data":"6332071a2064e033a77a420baebf45e0eedb3dd84a5c194c738c3fa1de734fba"}
Oct 30 20:49:28.372173 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.372136    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:28.760093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.760063    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:28.846904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.846870    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-xks-ca\") pod \"2415084f-ae27-4510-83e0-5c84adca4ad6\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") "
Oct 30 20:49:28.847102 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.846936    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-openshift-ca\") pod \"2415084f-ae27-4510-83e0-5c84adca4ad6\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") "
Oct 30 20:49:28.847102 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.847056    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2415084f-ae27-4510-83e0-5c84adca4ad6-kube-api-access-bwtxw\" (UniqueName: \"kubernetes.io/projected/2415084f-ae27-4510-83e0-5c84adca4ad6-kube-api-access-bwtxw\") pod \"2415084f-ae27-4510-83e0-5c84adca4ad6\" (UID: \"2415084f-ae27-4510-83e0-5c84adca4ad6\") "
Oct 30 20:49:28.847380 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.847354    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-openshift-ca" pod "2415084f-ae27-4510-83e0-5c84adca4ad6" (UID: "2415084f-ae27-4510-83e0-5c84adca4ad6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:49:28.847541 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.847522    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:49:28.849070 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.849044    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-xks-ca" pod "2415084f-ae27-4510-83e0-5c84adca4ad6" (UID: "2415084f-ae27-4510-83e0-5c84adca4ad6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:49:28.849165 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.849046    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2415084f-ae27-4510-83e0-5c84adca4ad6-kube-api-access-bwtxw" pod "2415084f-ae27-4510-83e0-5c84adca4ad6" (UID: "2415084f-ae27-4510-83e0-5c84adca4ad6"). InnerVolumeSpecName "kube-api-access-bwtxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:49:28.949052 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.948976    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2415084f-ae27-4510-83e0-5c84adca4ad6-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:49:28.949052 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:28.949004    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwtxw\" (UniqueName: \"kubernetes.io/projected/2415084f-ae27-4510-83e0-5c84adca4ad6-kube-api-access-bwtxw\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:49:29.284547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:29.284515    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:29.608610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:29.608526    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"
Oct 30 20:49:29.608610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:29.608552    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46" event={"ID":"2415084f-ae27-4510-83e0-5c84adca4ad6","Type":"ContainerDied","Data":"6d37b7e14702e25f179582107096957803a87b91cd6f1851c5f2a72d5307bd80"}
Oct 30 20:49:29.608610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:29.608603    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d37b7e14702e25f179582107096957803a87b91cd6f1851c5f2a72d5307bd80"
Oct 30 20:49:30.213428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:30.213366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:31.168832 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:31.168799    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:32.149166 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:32.149133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:33.055259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:33.055227    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:33.965674 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:33.965638    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:34.848294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:34.848257    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:36.105539 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:36.105495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:36.975837 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:36.975798    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:37.872936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:37.872890    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:38.661111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:38.661078    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:39.559002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:39.558965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:40.542472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:40.542428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:41.447624 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.447591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:41.717197 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.717116    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"]
Oct 30 20:49:41.717631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.717616    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2415084f-ae27-4510-83e0-5c84adca4ad6" containerName="revoke-keys"
Oct 30 20:49:41.717684 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.717633    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2415084f-ae27-4510-83e0-5c84adca4ad6" containerName="revoke-keys"
Oct 30 20:49:41.717736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.717727    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2415084f-ae27-4510-83e0-5c84adca4ad6" containerName="revoke-keys"
Oct 30 20:49:41.721216 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.721195    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.728145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.728052    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"e2e-conflict-a-b39ef5b3-kserve-self-signed-certs\""
Oct 30 20:49:41.728959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.728936    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"default-dockercfg-s2c6d\""
Oct 30 20:49:41.729036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.728962    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"kube-root-ca.crt\""
Oct 30 20:49:41.730825 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.730809    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"openshift-service-ca.crt\""
Oct 30 20:49:41.808813 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.808782    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"]
Oct 30 20:49:41.813997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.813968    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-model-cache\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.814161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.814102    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79cx6\" (UniqueName: \"kubernetes.io/projected/0657c174-450e-41e8-b3f5-9b7fe0474d18-kube-api-access-79cx6\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.814233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.814154    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0657c174-450e-41e8-b3f5-9b7fe0474d18-tls-certs\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.814233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.814190    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-tmp-dir\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.814343 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.814295    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-home\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.814438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.814398    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-dshm\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.863330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.863302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:41.915007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.914965    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-home\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915053    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-dshm\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915114    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-model-cache\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915314 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915194    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79cx6\" (UniqueName: \"kubernetes.io/projected/0657c174-450e-41e8-b3f5-9b7fe0474d18-kube-api-access-79cx6\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915314 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915230    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0657c174-450e-41e8-b3f5-9b7fe0474d18-tls-certs\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915314 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915258    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-tmp-dir\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915517 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915445    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-home\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915529    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-model-cache\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.915659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.915637    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-tmp-dir\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.917236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.917210    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-dshm\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.917608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.917590    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0657c174-450e-41e8-b3f5-9b7fe0474d18-tls-certs\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:41.924880 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:41.924856    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79cx6\" (UniqueName: \"kubernetes.io/projected/0657c174-450e-41e8-b3f5-9b7fe0474d18-kube-api-access-79cx6\") pod \"e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") " pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:42.033114 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:42.033075    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:42.352591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:42.352564    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"]
Oct 30 20:49:42.355479 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:42.355443    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0657c174_450e_41e8_b3f5_9b7fe0474d18.slice/crio-2f919f09b54288452cd7e3de4bd72c5dd8d6bee4a6af95956ad13be2e018715f WatchSource:0}: Error finding container 2f919f09b54288452cd7e3de4bd72c5dd8d6bee4a6af95956ad13be2e018715f: Status 404 returned error can't find the container with id 2f919f09b54288452cd7e3de4bd72c5dd8d6bee4a6af95956ad13be2e018715f
Oct 30 20:49:42.698368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:42.698269    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" event={"ID":"0657c174-450e-41e8-b3f5-9b7fe0474d18","Type":"ContainerStarted","Data":"fb2fb0f9f11b13229770a7c6522df9c7a849da1b97b5692753d17ed0059f1856"}
Oct 30 20:49:42.698957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:42.698932    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" event={"ID":"0657c174-450e-41e8-b3f5-9b7fe0474d18","Type":"ContainerStarted","Data":"2f919f09b54288452cd7e3de4bd72c5dd8d6bee4a6af95956ad13be2e018715f"}
Oct 30 20:49:42.719597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:42.719542    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podStartSLOduration=1.719527152 podStartE2EDuration="1.719527152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:42.718808705 +0000 UTC m=+690.833542895" watchObservedRunningTime="2026-10-30 20:49:42.719527152 +0000 UTC m=+690.834261341"
Oct 30 20:49:42.791575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:42.791532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:43.033368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:43.033335    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:43.651014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:43.650968    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:44.451576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:44.451540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:45.525349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:45.525314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:46.349797 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:46.349764    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:47.270374 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:47.270342    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:48.352871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.352832    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"]
Oct 30 20:49:48.356597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.356578    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.365348 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.365315    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-qn9bd\""
Oct 30 20:49:48.366256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.366231    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 20:49:48.366424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.366317    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 20:49:48.366424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.366377    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 20:49:48.366554 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.366526    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 20:49:48.367506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.367483    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-28159f2b53e5\""
Oct 30 20:49:48.367663 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.367638    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"]
Oct 30 20:49:48.376014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.375980    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:48.485912 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.485871    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf8d2\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-kube-api-access-zf8d2\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.485912 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.485916    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-routing-overlay-volume\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.486156 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.485997    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-provider-credentials\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.486156 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.486143    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-plugins-config-volume\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.587565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.587526    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-plugins-config-volume\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.587771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.587608    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf8d2\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-kube-api-access-zf8d2\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.587771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.587632    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-routing-overlay-volume\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.587771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.587659    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-provider-credentials\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.588288 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.588259    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-routing-overlay-volume\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.588478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.588456    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-plugins-config-volume\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.590193 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.590172    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-provider-credentials\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.603752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.603690    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf8d2\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-kube-api-access-zf8d2\") pod \"payload-processing-external-model-656f4b7695-wg98h\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") " pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.671986 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.671948    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:48.955900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:48.955872    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"]
Oct 30 20:49:48.957475 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:48.957450    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb6ef78_5f20_440d_936a_6d0de0a0723c.slice/crio-106c899f5f6f31158c8681e25e0b62d40b44e5bb079887bc77aef82d6d177477 WatchSource:0}: Error finding container 106c899f5f6f31158c8681e25e0b62d40b44e5bb079887bc77aef82d6d177477: Status 404 returned error can't find the container with id 106c899f5f6f31158c8681e25e0b62d40b44e5bb079887bc77aef82d6d177477
Oct 30 20:49:49.739379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:49.739343    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:49.756581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:49.756414    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h" event={"ID":"edb6ef78-5f20-440d-936a-6d0de0a0723c","Type":"ContainerStarted","Data":"d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e"}
Oct 30 20:49:49.756581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:49.756484    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h" event={"ID":"edb6ef78-5f20-440d-936a-6d0de0a0723c","Type":"ContainerStarted","Data":"106c899f5f6f31158c8681e25e0b62d40b44e5bb079887bc77aef82d6d177477"}
Oct 30 20:49:49.756581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:49.756539    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:49:49.780015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:49.779955    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h" podStartSLOduration=1.779941199 podStartE2EDuration="1.779941199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:49.776350815 +0000 UTC m=+697.891085017" watchObservedRunningTime="2026-10-30 20:49:49.779941199 +0000 UTC m=+697.894675388"
Oct 30 20:49:50.516871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:50.516838    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:50.831205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:50.831116    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:51.557144 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:49:51.557108    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:49:51.750078 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:51.750040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:52.033709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.033671    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:52.046901 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.046870    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:52.379218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.379128    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"]
Oct 30 20:49:52.383670 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.383638    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.402060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.402024    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"]
Oct 30 20:49:52.409513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.409484    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"]
Oct 30 20:49:52.413367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.413345    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.427862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.427831    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"]
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.434814    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435301    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435376    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435495    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435525    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435646    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435695    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz8lk\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-kube-api-access-zz8lk\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435726    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.435831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.435750    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81e9db27-0286-4be5-aa15-22c15b7183cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.536585 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536553    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.536585 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536589    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.536851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536615    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.536851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536655    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.536851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536676    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbk8p\" (UniqueName: \"kubernetes.io/projected/d2ea773e-9dd0-4894-8610-4c5c8f465087-kube-api-access-jbk8p\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.536851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536706    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.536851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536725    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.536851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536850    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2ea773e-9dd0-4894-8610-4c5c8f465087-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.537085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536971    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz8lk\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-kube-api-access-zz8lk\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.536989    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537007    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.537085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537036    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81e9db27-0286-4be5-aa15-22c15b7183cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537036    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537045    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537169    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.537379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537237    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537267    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537316    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537353    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.537631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537447    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.537631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537606    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537683    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.537801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.537778    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81e9db27-0286-4be5-aa15-22c15b7183cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.539196 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.539168    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.539592 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.539573    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.547271 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.547249    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.547452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.547431    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz8lk\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-kube-api-access-zz8lk\") pod \"maas-default-gateway-openshift-default-77f556d647-6nj94\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.639043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.638953    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.638992    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbk8p\" (UniqueName: \"kubernetes.io/projected/d2ea773e-9dd0-4894-8610-4c5c8f465087-kube-api-access-jbk8p\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639023    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639045    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2ea773e-9dd0-4894-8610-4c5c8f465087-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639093    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639143    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639211    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639264    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639314    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639452    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639529    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639735    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639816 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639780    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.639995 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.639967    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2ea773e-9dd0-4894-8610-4c5c8f465087-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.641805 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.641777    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.641906 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.641849    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.647976 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.647954    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2ea773e-9dd0-4894-8610-4c5c8f465087-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.648183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.648161    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbk8p\" (UniqueName: \"kubernetes.io/projected/d2ea773e-9dd0-4894-8610-4c5c8f465087-kube-api-access-jbk8p\") pod \"maas-default-gateway-openshift-default-77f556d647-b8h86\" (UID: \"d2ea773e-9dd0-4894-8610-4c5c8f465087\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.701362 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.701322    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:52.748427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.747690    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:52.757340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.757302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:52.808593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:52.808316    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:49:53.004647 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.004622    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"]
Oct 30 20:49:53.005475 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:53.005445    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e9db27_0286_4be5_aa15_22c15b7183cf.slice/crio-933c49e13db48b40bb8cd99964df58e36744c9db6578d05206e3b3f393087098 WatchSource:0}: Error finding container 933c49e13db48b40bb8cd99964df58e36744c9db6578d05206e3b3f393087098: Status 404 returned error can't find the container with id 933c49e13db48b40bb8cd99964df58e36744c9db6578d05206e3b3f393087098
Oct 30 20:49:53.008366 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.008336    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:53.008476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.008415    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:53.008476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.008444    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:53.055731 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.055702    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"]
Oct 30 20:49:53.057054 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:49:53.057026    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ea773e_9dd0_4894_8610_4c5c8f465087.slice/crio-4ffd70bcd9b8628058a14743c3efaa35e4713a4c2af8117ae223497a32b2d542 WatchSource:0}: Error finding container 4ffd70bcd9b8628058a14743c3efaa35e4713a4c2af8117ae223497a32b2d542: Status 404 returned error can't find the container with id 4ffd70bcd9b8628058a14743c3efaa35e4713a4c2af8117ae223497a32b2d542
Oct 30 20:49:53.059564 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.059526    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:53.059649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.059591    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:53.059649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.059620    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:49:53.752766 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.752732    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:53.805899 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.804938    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" event={"ID":"81e9db27-0286-4be5-aa15-22c15b7183cf","Type":"ContainerStarted","Data":"a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f"}
Oct 30 20:49:53.805899 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.805008    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" event={"ID":"81e9db27-0286-4be5-aa15-22c15b7183cf","Type":"ContainerStarted","Data":"933c49e13db48b40bb8cd99964df58e36744c9db6578d05206e3b3f393087098"}
Oct 30 20:49:53.812681 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.812639    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" event={"ID":"d2ea773e-9dd0-4894-8610-4c5c8f465087","Type":"ContainerStarted","Data":"4f579a20cc233d37c8ce762c6de20fb9e99797d47ceef672e78882ff41c867b5"}
Oct 30 20:49:53.812856 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.812693    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" event={"ID":"d2ea773e-9dd0-4894-8610-4c5c8f465087","Type":"ContainerStarted","Data":"4ffd70bcd9b8628058a14743c3efaa35e4713a4c2af8117ae223497a32b2d542"}
Oct 30 20:49:53.835102 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.835041    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podStartSLOduration=1.8350239099999999 podStartE2EDuration="1.83502391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:53.829899695 +0000 UTC m=+701.944633885" watchObservedRunningTime="2026-10-30 20:49:53.83502391 +0000 UTC m=+701.949758101"
Oct 30 20:49:53.861621 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:53.861547    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podStartSLOduration=1.861524389 podStartE2EDuration="1.861524389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:53.856768652 +0000 UTC m=+701.971502842" watchObservedRunningTime="2026-10-30 20:49:53.861524389 +0000 UTC m=+701.976258583"
Oct 30 20:49:54.230407 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.227967    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"]
Oct 30 20:49:54.235474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.235438    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:54.239673 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.239644    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"]
Oct 30 20:49:54.239831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.239736    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.244625 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.244596    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-f9a819\"/\"distinct-w3-f9a819-kserve-self-signed-certs\""
Oct 30 20:49:54.333310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.333271    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cd2330-9d8e-4241-940b-af257ca600bc-tls-certs\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.333310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.333322    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwrnc\" (UniqueName: \"kubernetes.io/projected/84cd2330-9d8e-4241-940b-af257ca600bc-kube-api-access-rwrnc\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.333610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.333547    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-dshm\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.333669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.333650    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-tmp-dir\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.333723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.333679    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-home\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.333891 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.333868    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-model-cache\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.435330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.435279    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cd2330-9d8e-4241-940b-af257ca600bc-tls-certs\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.435524 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.435355    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwrnc\" (UniqueName: \"kubernetes.io/projected/84cd2330-9d8e-4241-940b-af257ca600bc-kube-api-access-rwrnc\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.435989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.435962    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-dshm\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.436124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.436061    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-tmp-dir\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.436124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.436089    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-home\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.436255 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.436165    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-model-cache\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.436657 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.436635    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-model-cache\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.437683 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.437658    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-tmp-dir\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.437909 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.437886    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-home\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.439317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.439285    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cd2330-9d8e-4241-940b-af257ca600bc-tls-certs\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.439722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.439698    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-dshm\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.446549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.446526    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwrnc\" (UniqueName: \"kubernetes.io/projected/84cd2330-9d8e-4241-940b-af257ca600bc-kube-api-access-rwrnc\") pod \"distinct-w3-f9a819-kserve-6dc58f668-lvczv\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") " pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.605299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.605258    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:54.701717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.701637    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:49:54.703630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.703603    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:49:54.703829 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.703810    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 20:49:54.748677 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.748649    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:49:54.776412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.758519    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:49:54.776412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.758585    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 20:49:54.900979 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.900876    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:54.948542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:54.947857    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"]
Oct 30 20:49:55.702825 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.702789    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:49:55.703248 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.702849    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 20:49:55.749037 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.749002    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:49:55.749188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.749056    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 20:49:55.832037 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.831991    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" event={"ID":"84cd2330-9d8e-4241-940b-af257ca600bc","Type":"ContainerStarted","Data":"931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0"}
Oct 30 20:49:55.832217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.832050    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" event={"ID":"84cd2330-9d8e-4241-940b-af257ca600bc","Type":"ContainerStarted","Data":"5e1dc41007c506fb4b262f063b0c1197daa31ec2892bb98a6560fa7f2a6dca58"}
Oct 30 20:49:55.875907 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.875867    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:55.879416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:55.878912    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podStartSLOduration=1.878893022 podStartE2EDuration="1.878893022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:49:55.87394493 +0000 UTC m=+703.988679177" watchObservedRunningTime="2026-10-30 20:49:55.878893022 +0000 UTC m=+703.993627212"
Oct 30 20:49:56.608634 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:56.605588    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:49:56.702412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:56.701810    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:49:56.702412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:56.701891    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 20:49:56.749935 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:56.749898    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:49:56.750564 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:56.750514    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 20:49:57.523410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:57.523290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:57.702412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:57.701853    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:49:57.702412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:57.701924    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 20:49:57.749411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:57.748702    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:49:57.749411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:57.748761    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 20:49:58.702411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:58.701861    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:49:58.702411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:58.701929    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 20:49:58.749067 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:58.749005    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:49:58.749428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:58.749348    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 20:49:59.348442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:59.348321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:49:59.705471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:59.703758    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:49:59.705471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:59.703822    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 20:49:59.748460 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:59.748425    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:49:59.748669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:49:59.748485    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 20:50:00.702472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:00.702427    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 20:50:00.702656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:00.702499    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 20:50:00.748448 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:00.748379    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 20:50:00.748838 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:00.748473    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 20:50:00.769054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:00.769025    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:50:01.452781 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.452748    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:01.741836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.741733    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"]
Oct 30 20:50:01.747297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.747261    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.764795 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.764755    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"]
Oct 30 20:50:01.780555 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.766838    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db7f461-eb75-4c63-960a-ed412a088428-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.784325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.784286    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.784668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.784637    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.784874 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.784853    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb6vj\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-kube-api-access-kb6vj\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.785070 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.785039    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.785296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.785264    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.785567 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.785546    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.785828 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.785809    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.786023 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.785989    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db7f461-eb75-4c63-960a-ed412a088428-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.888036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.887991    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.888429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.888362    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb6vj\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-kube-api-access-kb6vj\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.888620 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.888598    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.888750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.888732    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.888869 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.888538    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.888945 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.888912    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.889173 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.889147    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.889260 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.889245    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.889322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.889292    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db7f461-eb75-4c63-960a-ed412a088428-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.889417 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.889375    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db7f461-eb75-4c63-960a-ed412a088428-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.889490 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.889446    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.889858 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.889838    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.890002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.889912    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.890324 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.890297    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db7f461-eb75-4c63-960a-ed412a088428-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.893238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.893217    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.894021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.894000    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db7f461-eb75-4c63-960a-ed412a088428-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.900508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.900470    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:01.900856 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:01.900775    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb6vj\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-kube-api-access-kb6vj\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:02.079353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.079312    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:02.416749 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.416704    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"]
Oct 30 20:50:02.418342 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.418301    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:02.418736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.418706    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:02.418902 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.418881    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:02.520474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.520438    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:50:02.522083 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.522054    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:50:02.523085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.523060    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86"
Oct 30 20:50:02.660742 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.660375    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:50:02.661642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.661592    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:50:02.662760 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.662738    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 20:50:02.917960 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:02.917859    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:03.125986 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:03.125948    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" event={"ID":"6db7f461-eb75-4c63-960a-ed412a088428","Type":"ContainerStarted","Data":"777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00"}
Oct 30 20:50:03.126195 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:03.126007    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" event={"ID":"6db7f461-eb75-4c63-960a-ed412a088428","Type":"ContainerStarted","Data":"903a7866d6a5ab325e3821eba5059da6add8bc25ab5e954e2be25b1276d03751"}
Oct 30 20:50:03.168636 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:03.167593    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podStartSLOduration=2.167569905 podStartE2EDuration="2.167569905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:50:03.165196404 +0000 UTC m=+711.279930595" watchObservedRunningTime="2026-10-30 20:50:03.167569905 +0000 UTC m=+711.282304094"
Oct 30 20:50:03.268422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:03.264102    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:04.086410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:04.085689    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:04.086410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:04.085819    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:50:04.086410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:04.085862    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 20:50:04.294413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:04.282344    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:04.610672 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:04.610630    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:50:04.640134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:04.640107    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:50:05.087412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:05.085443    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:50:05.087412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:05.085520    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 20:50:05.240413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:05.240281    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:50:05.426464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:05.424463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:06.080125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:06.080086    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:50:06.080329 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:06.080150    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 20:50:06.469361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:06.469270    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:07.080243 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:07.080205    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:50:07.080473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:07.080276    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 20:50:07.241306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:07.241248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:07.679026 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:07.678971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:08.088484 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:08.080416    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:50:08.088484 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:08.080477    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 20:50:08.796531 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:08.796495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:09.097954 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:09.097856    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 20:50:09.098232 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:09.098192    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 20:50:10.169409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:10.169345    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:11.136411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:11.135524    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:50:11.136411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:11.135623    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:11.194612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:11.194558    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:11.792410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:11.792273    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:11.840411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:11.838466    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:12.048932 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:12.045778    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:12.114669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:12.114631    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:12.116162 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:12.116141    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:12.130593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:12.130563    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 20:50:13.039559 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:13.039518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:13.857979 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:13.857940    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:14.755958 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:14.755924    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:15.563198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:15.563160    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:16.632465 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:16.632431    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:16.941905 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:16.941809    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"]
Oct 30 20:50:17.207426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:17.207305    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:17.686867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:17.686830    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:17.959698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:17.959613    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:18.654461 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:18.654420    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:19.540444 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:19.540406    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:20.540495 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:20.540456    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:21.401143 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:21.401107    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:21.558749 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:50:21.558708    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:50:21.942361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:21.942321    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h" podUID="edb6ef78-5f20-440d-936a-6d0de0a0723c" containerName="payload-processing" containerID="cri-o://d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e" gracePeriod=25
Oct 30 20:50:22.037587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.037555    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:22.148223 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.148141    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:22.208369 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.208303    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:50:22.274471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.273538    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-kube-api-access-zf8d2\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-kube-api-access-zf8d2\") pod \"edb6ef78-5f20-440d-936a-6d0de0a0723c\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") "
Oct 30 20:50:22.274471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.273701    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-plugins-config-volume\") pod \"edb6ef78-5f20-440d-936a-6d0de0a0723c\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") "
Oct 30 20:50:22.274471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.273753    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-provider-credentials\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-provider-credentials\") pod \"edb6ef78-5f20-440d-936a-6d0de0a0723c\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") "
Oct 30 20:50:22.274471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.273861    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-routing-overlay-volume\") pod \"edb6ef78-5f20-440d-936a-6d0de0a0723c\" (UID: \"edb6ef78-5f20-440d-936a-6d0de0a0723c\") "
Oct 30 20:50:22.275570 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.275493    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-routing-overlay-volume" pod "edb6ef78-5f20-440d-936a-6d0de0a0723c" (UID: "edb6ef78-5f20-440d-936a-6d0de0a0723c"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:50:22.277124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.277100    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-plugins-config-volume" pod "edb6ef78-5f20-440d-936a-6d0de0a0723c" (UID: "edb6ef78-5f20-440d-936a-6d0de0a0723c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:50:22.296993 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.296933    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-kube-api-access-zf8d2" pod "edb6ef78-5f20-440d-936a-6d0de0a0723c" (UID: "edb6ef78-5f20-440d-936a-6d0de0a0723c"). InnerVolumeSpecName "kube-api-access-zf8d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:50:22.308983 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.308939    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-provider-credentials" pod "edb6ef78-5f20-440d-936a-6d0de0a0723c" (UID: "edb6ef78-5f20-440d-936a-6d0de0a0723c"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:50:22.381104 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.380822    4852 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-routing-overlay-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:22.381405 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.381367    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf8d2\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-kube-api-access-zf8d2\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:22.381640 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.381541    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edb6ef78-5f20-440d-936a-6d0de0a0723c-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:22.381882 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.381847    4852 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edb6ef78-5f20-440d-936a-6d0de0a0723c-provider-credentials\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:22.426004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.425776    4852 generic.go:356] "Generic (PLEG): container finished" podID="edb6ef78-5f20-440d-936a-6d0de0a0723c" containerID="d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e" exitCode=0
Oct 30 20:50:22.426004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.425824    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h" event={"ID":"edb6ef78-5f20-440d-936a-6d0de0a0723c","Type":"ContainerDied","Data":"d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e"}
Oct 30 20:50:22.426004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.425865    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h" event={"ID":"edb6ef78-5f20-440d-936a-6d0de0a0723c","Type":"ContainerDied","Data":"106c899f5f6f31158c8681e25e0b62d40b44e5bb079887bc77aef82d6d177477"}
Oct 30 20:50:22.426004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.425853    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"
Oct 30 20:50:22.426004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.425891    4852 scope.go:122] "RemoveContainer" containerID="d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e"
Oct 30 20:50:22.438622 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.438592    4852 scope.go:122] "RemoveContainer" containerID="d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e"
Oct 30 20:50:22.438980 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:50:22.438960    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e\": container with ID starting with d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e not found: ID does not exist" containerID="d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e"
Oct 30 20:50:22.439038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.438990    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e"} err="failed to get container status \"d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e\": rpc error: code = NotFound desc = could not find container \"d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e\": container with ID starting with d80dfee92a0b7f4616b4771488f1efee0a7db2100c413d0250631cf1d060530e not found: ID does not exist"
Oct 30 20:50:22.453263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.453211    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"]
Oct 30 20:50:22.456939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.456912    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-656f4b7695-wg98h"]
Oct 30 20:50:22.652880 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.652847    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"]
Oct 30 20:50:22.653472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.653454    4852 cpu_manager.go:414] "containerMap: removing container" podUID="edb6ef78-5f20-440d-936a-6d0de0a0723c" containerName="payload-processing"
Oct 30 20:50:22.653472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.653472    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edb6ef78-5f20-440d-936a-6d0de0a0723c" containerName="payload-processing"
Oct 30 20:50:22.653607 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.653550    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="edb6ef78-5f20-440d-936a-6d0de0a0723c" containerName="payload-processing"
Oct 30 20:50:22.658038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.658016    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.666525 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666498    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 20:50:22.666709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666499    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-7d1c765a6192\""
Oct 30 20:50:22.666709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666499    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 20:50:22.666709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666533    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-qn9bd\""
Oct 30 20:50:22.666938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666558    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 20:50:22.666938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666579    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-551b354a5db7\""
Oct 30 20:50:22.666938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666609    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 20:50:22.666938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.666678    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-e75e693dc1a7\""
Oct 30 20:50:22.669238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.669214    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"]
Oct 30 20:50:22.689200 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.689160    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:22.787746 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.787711    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-plugins-config-volume\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.787938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.787897    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-routing-overlay-volume\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.788007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.787993    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-provider-credentials\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.788054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.788019    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszzk\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-kube-api-access-tszzk\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.888925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.888868    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-routing-overlay-volume\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.889139 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.888957    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-provider-credentials\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.889139 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.888982    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszzk\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-kube-api-access-tszzk\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.889139 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.889086    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-plugins-config-volume\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.889698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.889668    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-routing-overlay-volume\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.889855 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.889835    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-plugins-config-volume\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.891640 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.891617    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-provider-credentials\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.901618 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.901590    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszzk\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-kube-api-access-tszzk\") pod \"payload-processing-external-model-57db8b5885-9ncq2\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") " pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:22.974236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:22.974155    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:23.265345 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:23.265310    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"]
Oct 30 20:50:23.265906 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:50:23.265880    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cab6d3e_a657_4e30_89aa_eef0a39c9886.slice/crio-9366c785c06db3f34936fd850ed0aa2c7ffde9b70a21a02fd1ccb76d0b54f6a1 WatchSource:0}: Error finding container 9366c785c06db3f34936fd850ed0aa2c7ffde9b70a21a02fd1ccb76d0b54f6a1: Status 404 returned error can't find the container with id 9366c785c06db3f34936fd850ed0aa2c7ffde9b70a21a02fd1ccb76d0b54f6a1
Oct 30 20:50:23.433616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:23.433582    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2" event={"ID":"8cab6d3e-a657-4e30-89aa-eef0a39c9886","Type":"ContainerStarted","Data":"c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631"}
Oct 30 20:50:23.433811 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:23.433643    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2" event={"ID":"8cab6d3e-a657-4e30-89aa-eef0a39c9886","Type":"ContainerStarted","Data":"9366c785c06db3f34936fd850ed0aa2c7ffde9b70a21a02fd1ccb76d0b54f6a1"}
Oct 30 20:50:23.433811 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:23.433783    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:23.470019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:23.469967    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2" podStartSLOduration=1.469950324 podStartE2EDuration="1.469950324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:50:23.467084239 +0000 UTC m=+731.581818430" watchObservedRunningTime="2026-10-30 20:50:23.469950324 +0000 UTC m=+731.584684551"
Oct 30 20:50:23.608578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:23.608493    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:24.004163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.004076    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edb6ef78-5f20-440d-936a-6d0de0a0723c" path="/var/lib/kubelet/pods/edb6ef78-5f20-440d-936a-6d0de0a0723c/volumes"
Oct 30 20:50:24.047111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.047073    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"]
Oct 30 20:50:24.051217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.051199    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.064942 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.064903    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"]
Oct 30 20:50:24.204752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.204709    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrnm4\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-kube-api-access-qrnm4\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.204752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.204754    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.205044 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.204809    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.205044 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.204991    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.205044 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.205032    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.205200 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.205151    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.205251 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.205209    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.205303 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.205262    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.205358 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.205339    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306122    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrnm4\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-kube-api-access-qrnm4\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306180    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306225    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306282    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306308    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306433    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306520    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306588    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306641    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306843 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306694    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.306903 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.306880    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.307160 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.307137    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.307233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.307163    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.307233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.307139    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.308710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.308684    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.309132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.309109    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.316446 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.316418    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.316600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.316581    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrnm4\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-kube-api-access-qrnm4\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.365975 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.365939    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:24.366217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.366129    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:24.677263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.677220    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"]
Oct 30 20:50:24.678220 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:50:24.678188    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c5027e_60b3_415f_a6d7_75e14d0a78f3.slice/crio-1b65743dff67eb0f983132535d3c1637a5d641057a821f8d83763989b4ec1a64 WatchSource:0}: Error finding container 1b65743dff67eb0f983132535d3c1637a5d641057a821f8d83763989b4ec1a64: Status 404 returned error can't find the container with id 1b65743dff67eb0f983132535d3c1637a5d641057a821f8d83763989b4ec1a64
Oct 30 20:50:24.680793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.680761    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:24.680861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.680828    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:24.680898 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:24.680859    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:25.141764 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:25.141706    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:25.481449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:25.481322    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" event={"ID":"d2c5027e-60b3-415f-a6d7-75e14d0a78f3","Type":"ContainerStarted","Data":"0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba"}
Oct 30 20:50:25.481449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:25.481380    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" event={"ID":"d2c5027e-60b3-415f-a6d7-75e14d0a78f3","Type":"ContainerStarted","Data":"1b65743dff67eb0f983132535d3c1637a5d641057a821f8d83763989b4ec1a64"}
Oct 30 20:50:25.512501 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:25.512445    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podStartSLOduration=1.5124286260000002 podStartE2EDuration="1.512428626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:50:25.509610576 +0000 UTC m=+733.624344766" watchObservedRunningTime="2026-10-30 20:50:25.512428626 +0000 UTC m=+733.627162878"
Oct 30 20:50:25.929784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:25.929741    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:26.373411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:26.367734    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:26.375569 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:26.375532    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:26.375697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:26.375627    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:26.729344 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:26.729254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:27.369943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:27.369904    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:27.370228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:27.370198    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:27.635239 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:27.635152    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:28.369412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:28.368798    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:28.369412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:28.368866    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:29.472413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:29.471788    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:29.490450 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:29.474848    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:29.490450 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:29.474910    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:29.841412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:29.840497    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:29.932410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:29.931489    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:50:30.003412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.002489    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:50:30.044413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.043504    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:30.150472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.149486    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:50:30.165410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.150711    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:50:30.186415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.180594    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:30.377427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.366670    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:30.377427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.366736    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:30.798227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.797497    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Oct 30 20:50:30.847501 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.847449    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:30.848125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.848099    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:50:30.954445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.932546    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:30.954445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:30.933491    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:50:31.070684 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.070531    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:31.142404 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.142358    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:50:31.149890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.149828    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:31.154501 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.154457    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:31.179906 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.163535    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:50:31.180240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.180218    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:50:31.188415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.184477    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:31.229414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.201841    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:50:31.418473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.410940    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:31.435523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.418841    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:31.455544 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.455493    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:31.849808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.848621    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:31.923844 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.866163    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.175486    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:31.951711    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.194341    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.013926    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.194457    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.198725    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.228692    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:32.272471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.226811    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:32.493411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.492554    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:32.493411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.492636    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:32.861545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:32.861502    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:50:33.079411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.078501    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:33.141123 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.141018    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:33.178596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.178547    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:33.204748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.204701    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:33.206163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.206130    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:50:33.236411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.231671    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:50:33.266725 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.266675    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:33.372805 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.372769    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 20:50:33.373131 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.373082    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 20:50:33.458799 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.458683    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:33.460048 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.460019    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:33.586188 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.586141    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:50:33.586539 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.586489    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:33.595316 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.595273    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:50:33.943411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.933786    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:50:33.978449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.976312    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:50:33.988940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:33.988912    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:50:34.003578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.003256    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:50:34.003578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.003310    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:50:34.107627 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.107580    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:50:34.465210 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.465171    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:50:34.655109 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.655072    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:34.810409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.809712    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:34.810409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.810086    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:34.811582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:34.811558    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 20:50:35.844002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:35.843962    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:36.832069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:36.832036    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:37.853125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:37.853089    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:38.337573 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:38.337538    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:38.867509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:38.867465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:39.854503 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:39.854473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:40.488808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:40.488773    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"]
Oct 30 20:50:40.538489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:40.538458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:40.888714 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:40.888649    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:41.076970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.076939    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"]
Oct 30 20:50:41.084199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.084172    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.106333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.106301    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"]
Oct 30 20:50:41.247295 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247191    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247295 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247267    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247356    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247463    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466e556f-2463-4275-a27f-23e56e536039-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247500    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247625 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247518    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466e556f-2463-4275-a27f-23e56e536039-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247651    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247716 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247693    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhd7w\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-kube-api-access-mhd7w\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.247750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.247724    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348435 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348367    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhd7w\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-kube-api-access-mhd7w\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348458    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348509    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348555    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348732 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348636    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348725    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466e556f-2463-4275-a27f-23e56e536039-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348778    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348811    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466e556f-2463-4275-a27f-23e56e536039-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348884    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.348936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348903    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.349038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.348993    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.349082 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.349035    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.349148 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.349127    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.349649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.349620    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466e556f-2463-4275-a27f-23e56e536039-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.351423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.351381    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.351520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.351485    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466e556f-2463-4275-a27f-23e56e536039-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.357839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.357816    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.357939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.357916    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhd7w\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-kube-api-access-mhd7w\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.397772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.397737    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:41.685906 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.685879    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"]
Oct 30 20:50:41.686794 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:50:41.686770    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod466e556f_2463_4275_a27f_23e56e536039.slice/crio-9c30a25efef1ac25e4ccaeba0f1e818a6a2250fe68d1100ffc8816240c245bd1 WatchSource:0}: Error finding container 9c30a25efef1ac25e4ccaeba0f1e818a6a2250fe68d1100ffc8816240c245bd1: Status 404 returned error can't find the container with id 9c30a25efef1ac25e4ccaeba0f1e818a6a2250fe68d1100ffc8816240c245bd1
Oct 30 20:50:41.689698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.689665    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:41.689764 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.689732    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:41.689801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.689765    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:41.844253 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:41.844219    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:42.143939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:42.143900    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" event={"ID":"466e556f-2463-4275-a27f-23e56e536039","Type":"ContainerStarted","Data":"9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2"}
Oct 30 20:50:42.144166 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:42.143964    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" event={"ID":"466e556f-2463-4275-a27f-23e56e536039","Type":"ContainerStarted","Data":"9c30a25efef1ac25e4ccaeba0f1e818a6a2250fe68d1100ffc8816240c245bd1"}
Oct 30 20:50:42.180352 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:42.180298    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podStartSLOduration=1.180284191 podStartE2EDuration="1.180284191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:50:42.179665704 +0000 UTC m=+750.294399895" watchObservedRunningTime="2026-10-30 20:50:42.180284191 +0000 UTC m=+750.295018380"
Oct 30 20:50:42.398001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:42.397914    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:42.399706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:42.399673    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:42.399866 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:42.399727    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:42.875614 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:42.875584    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:43.399229 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:43.399186    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:43.399420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:43.399258    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:43.872411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:43.868550    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:44.398771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:44.398731    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:44.399165 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:44.398815    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:44.736838 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:44.736745    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:45.399075 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:45.399031    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:45.399529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:45.399108    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:46.399410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:46.398848    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:46.399410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:46.398915    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:47.006317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.006277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:47.023534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.023484    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:47.160407 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.159111    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:47.206530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.206478    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:47.207301 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.207269    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:47.280762 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.280664    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:50:47.399660 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.399179    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:47.399660 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.399244    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:47.605597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.605488    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:47.987626 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:47.987494    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:48.028410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.027505    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:48.053412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.053099    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:50:48.194429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.175922    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Oct 30 20:50:48.194429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.178075    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:50:48.351442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.261498    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:48.351442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.281432    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:48.351442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.347110    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:50:48.351442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.347175    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:50:48.384420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.378587    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:50:48.394794 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.389809    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:50:48.404576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.404534    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:48.408415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.405125    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:48.606947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.606614    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:48.608598 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:48.607904    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:50:49.033414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.030777    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:49.052413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.050952    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:50:49.086411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.053484    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:49.148139 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.148096    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": context deadline exceeded"
Oct 30 20:50:49.246145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.182527    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:49.265643 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.265597    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:50:49.295507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.293623    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:49.421414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.396793    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:49.426225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.404102    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:49.426225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.405223    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:49.426225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.426145    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:49.607752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:49.607648    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:50.065612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.065541    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:50:50.074420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.066799    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": context deadline exceeded"
Oct 30 20:50:50.134102 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.134048    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:50:50.254553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.254469    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:50.269451 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.269364    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:50.274781 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.274738    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:50:50.457469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.420668    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:50.457469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.420732    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:50.531411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.523490    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:50.601411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.590357    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:50:50.603294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.602115    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:50.626409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:50.623124    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:51.063153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.063108    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:51.109421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.066026    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:50:51.109421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.067642    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:50:51.148409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.136475    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:50:51.179657 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.166986    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:50:51.333457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.332351    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:50:51.347512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.347164    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:50:51.398613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.398576    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 20:50:51.398810 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.398643    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 20:50:51.551650 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:51.551608    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:51.571574 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:50:51.571535    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:50:52.398372 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.398336    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"]
Oct 30 20:50:52.402712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.402689    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.411894 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.411870    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:50:52.412057 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.412040    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:50:52.416121 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.416091    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.416238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.416149    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h8rk\" (UniqueName: \"kubernetes.io/projected/ec91dca4-c220-48e7-af17-a5c1780eabda-kube-api-access-8h8rk\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.416299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.416255    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.427333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.427298    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"]
Oct 30 20:50:52.517890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.517835    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.518106 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.517906    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h8rk\" (UniqueName: \"kubernetes.io/projected/ec91dca4-c220-48e7-af17-a5c1780eabda-kube-api-access-8h8rk\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.518106 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.518036    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.524026 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.523963    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.526155 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.526126    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.534054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.534016    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h8rk\" (UniqueName: \"kubernetes.io/projected/ec91dca4-c220-48e7-af17-a5c1780eabda-kube-api-access-8h8rk\") pod \"maas-api-revoke-sub-5fb76500-tgqhs\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.545861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.545829    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:52.719643 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.719561    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:52.931630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.931592    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:52.932501 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.932468    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:52.949803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:52.949777    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:50:53.024434 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:53.022995    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"]
Oct 30 20:50:53.025734 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:50:53.025686    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec91dca4_c220_48e7_af17_a5c1780eabda.slice/crio-a13569e6871fcc07c3d7350c436752f9c7ee52c309b88d0ac444e280da460d4b WatchSource:0}: Error finding container a13569e6871fcc07c3d7350c436752f9c7ee52c309b88d0ac444e280da460d4b: Status 404 returned error can't find the container with id a13569e6871fcc07c3d7350c436752f9c7ee52c309b88d0ac444e280da460d4b
Oct 30 20:50:53.441585 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:53.441489    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:53.736432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:53.736321    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs" event={"ID":"ec91dca4-c220-48e7-af17-a5c1780eabda","Type":"ContainerStarted","Data":"c6c9606d1e5a348cebe7f079eb730c9ec8324cfc74b2bf355780e60e893dbe15"}
Oct 30 20:50:53.736432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:53.736378    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs" event={"ID":"ec91dca4-c220-48e7-af17-a5c1780eabda","Type":"ContainerStarted","Data":"a13569e6871fcc07c3d7350c436752f9c7ee52c309b88d0ac444e280da460d4b"}
Oct 30 20:50:53.763033 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:53.762981    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs" podStartSLOduration=1.762966981 podStartE2EDuration="1.762966981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:50:53.760894632 +0000 UTC m=+761.875628823" watchObservedRunningTime="2026-10-30 20:50:53.762966981 +0000 UTC m=+761.877701170"
Oct 30 20:50:54.749834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:54.749796    4852 generic.go:356] "Generic (PLEG): container finished" podID="ec91dca4-c220-48e7-af17-a5c1780eabda" containerID="c6c9606d1e5a348cebe7f079eb730c9ec8324cfc74b2bf355780e60e893dbe15" exitCode=0
Oct 30 20:50:54.750346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:54.749891    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs" event={"ID":"ec91dca4-c220-48e7-af17-a5c1780eabda","Type":"ContainerDied","Data":"c6c9606d1e5a348cebe7f079eb730c9ec8324cfc74b2bf355780e60e893dbe15"}
Oct 30 20:50:54.780299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:54.780262    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:55.509315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.509271    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main" containerID="cri-o://fb2fb0f9f11b13229770a7c6522df9c7a849da1b97b5692753d17ed0059f1856" gracePeriod=45
Oct 30 20:50:55.608966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.608886    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:55.758367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.758332    4852 generic.go:356] "Generic (PLEG): container finished" podID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerID="fb2fb0f9f11b13229770a7c6522df9c7a849da1b97b5692753d17ed0059f1856" exitCode=0
Oct 30 20:50:55.758772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.758439    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" event={"ID":"0657c174-450e-41e8-b3f5-9b7fe0474d18","Type":"ContainerDied","Data":"fb2fb0f9f11b13229770a7c6522df9c7a849da1b97b5692753d17ed0059f1856"}
Oct 30 20:50:55.796499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.796468    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:50:55.862256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862223    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-home\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-home\") pod \"0657c174-450e-41e8-b3f5-9b7fe0474d18\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") "
Oct 30 20:50:55.862432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862318    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-model-cache\") pod \"0657c174-450e-41e8-b3f5-9b7fe0474d18\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") "
Oct 30 20:50:55.862432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862423    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-dshm\") pod \"0657c174-450e-41e8-b3f5-9b7fe0474d18\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") "
Oct 30 20:50:55.862574 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862465    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0657c174-450e-41e8-b3f5-9b7fe0474d18-tls-certs\" (UniqueName: \"kubernetes.io/secret/0657c174-450e-41e8-b3f5-9b7fe0474d18-tls-certs\") pod \"0657c174-450e-41e8-b3f5-9b7fe0474d18\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") "
Oct 30 20:50:55.862574 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862503    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0657c174-450e-41e8-b3f5-9b7fe0474d18-kube-api-access-79cx6\" (UniqueName: \"kubernetes.io/projected/0657c174-450e-41e8-b3f5-9b7fe0474d18-kube-api-access-79cx6\") pod \"0657c174-450e-41e8-b3f5-9b7fe0474d18\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") "
Oct 30 20:50:55.862680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862594    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-tmp-dir\") pod \"0657c174-450e-41e8-b3f5-9b7fe0474d18\" (UID: \"0657c174-450e-41e8-b3f5-9b7fe0474d18\") "
Oct 30 20:50:55.862680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862617    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-home" pod "0657c174-450e-41e8-b3f5-9b7fe0474d18" (UID: "0657c174-450e-41e8-b3f5-9b7fe0474d18"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:50:55.862680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862669    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-model-cache" pod "0657c174-450e-41e8-b3f5-9b7fe0474d18" (UID: "0657c174-450e-41e8-b3f5-9b7fe0474d18"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:50:55.862917 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.862892    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-tmp-dir" pod "0657c174-450e-41e8-b3f5-9b7fe0474d18" (UID: "0657c174-450e-41e8-b3f5-9b7fe0474d18"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:50:55.863133 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.863116    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:55.863199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.863139    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:55.863199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.863152    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:55.865565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.865533    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0657c174-450e-41e8-b3f5-9b7fe0474d18-tls-certs" pod "0657c174-450e-41e8-b3f5-9b7fe0474d18" (UID: "0657c174-450e-41e8-b3f5-9b7fe0474d18"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:50:55.866443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.866419    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-dshm" pod "0657c174-450e-41e8-b3f5-9b7fe0474d18" (UID: "0657c174-450e-41e8-b3f5-9b7fe0474d18"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:50:55.866565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.866531    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0657c174-450e-41e8-b3f5-9b7fe0474d18-kube-api-access-79cx6" pod "0657c174-450e-41e8-b3f5-9b7fe0474d18" (UID: "0657c174-450e-41e8-b3f5-9b7fe0474d18"). InnerVolumeSpecName "kube-api-access-79cx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:50:55.900234 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.900206    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:55.964604 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.964561    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-openshift-ca\") pod \"ec91dca4-c220-48e7-af17-a5c1780eabda\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") "
Oct 30 20:50:55.964778 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.964643    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-xks-ca\") pod \"ec91dca4-c220-48e7-af17-a5c1780eabda\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") "
Oct 30 20:50:55.964778 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.964681    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec91dca4-c220-48e7-af17-a5c1780eabda-kube-api-access-8h8rk\" (UniqueName: \"kubernetes.io/projected/ec91dca4-c220-48e7-af17-a5c1780eabda-kube-api-access-8h8rk\") pod \"ec91dca4-c220-48e7-af17-a5c1780eabda\" (UID: \"ec91dca4-c220-48e7-af17-a5c1780eabda\") "
Oct 30 20:50:55.965015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.964991    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-openshift-ca" pod "ec91dca4-c220-48e7-af17-a5c1780eabda" (UID: "ec91dca4-c220-48e7-af17-a5c1780eabda"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:50:55.965256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.965234    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:55.965326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.965261    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0657c174-450e-41e8-b3f5-9b7fe0474d18-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:55.965326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.965276    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0657c174-450e-41e8-b3f5-9b7fe0474d18-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:55.965326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.965290    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79cx6\" (UniqueName: \"kubernetes.io/projected/0657c174-450e-41e8-b3f5-9b7fe0474d18-kube-api-access-79cx6\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:55.966868 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.966837    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-xks-ca" pod "ec91dca4-c220-48e7-af17-a5c1780eabda" (UID: "ec91dca4-c220-48e7-af17-a5c1780eabda"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:50:55.966956 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:55.966923    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec91dca4-c220-48e7-af17-a5c1780eabda-kube-api-access-8h8rk" pod "ec91dca4-c220-48e7-af17-a5c1780eabda" (UID: "ec91dca4-c220-48e7-af17-a5c1780eabda"). InnerVolumeSpecName "kube-api-access-8h8rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:50:56.066201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.066170    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ec91dca4-c220-48e7-af17-a5c1780eabda-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:56.066201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.066200    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h8rk\" (UniqueName: \"kubernetes.io/projected/ec91dca4-c220-48e7-af17-a5c1780eabda-kube-api-access-8h8rk\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:50:56.134626 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.134562    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"]
Oct 30 20:50:56.135372 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.135346    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ec91dca4-c220-48e7-af17-a5c1780eabda" containerName="revoke-keys"
Oct 30 20:50:56.135520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.135374    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec91dca4-c220-48e7-af17-a5c1780eabda" containerName="revoke-keys"
Oct 30 20:50:56.135520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.135447    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main"
Oct 30 20:50:56.135520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.135456    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main"
Oct 30 20:50:56.135612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.135585    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec91dca4-c220-48e7-af17-a5c1780eabda" containerName="revoke-keys"
Oct 30 20:50:56.135612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.135602    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" containerName="main"
Oct 30 20:50:56.139530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.139508    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.166936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.166908    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.166957    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167178 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.167157    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167231 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.167205    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167272 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.167262    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167358 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.167346    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44mx5\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-kube-api-access-44mx5\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.167381    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.167448    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.167505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.167486    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.174354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.174327    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"]
Oct 30 20:50:56.269144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269052    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269121    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269154    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269172    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269236    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44mx5\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-kube-api-access-44mx5\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269268    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269314    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269360    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269437    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269577    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-certs\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269605    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-credential-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.269889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269851    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-socket\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.270021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.269925    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-data\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.270129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.270111    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istiod-ca-cert\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.271668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.271642    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-envoy\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.271880 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.271860    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-podinfo\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.278118 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.278090    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-token\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.278307 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.278289    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44mx5\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-kube-api-access-44mx5\") pod \"e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") " pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.453561 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.453529    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:56.675077 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.675041    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:56.757755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.757719    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"]
Oct 30 20:50:56.757904 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:50:56.757782    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf04b00d_07bc_453d_a7fa_2a8622fb5a0c.slice/crio-b31c642c300cf6ecbf8dfb42710cc52ed9ab205232f47f28e09c48719922d64e WatchSource:0}: Error finding container b31c642c300cf6ecbf8dfb42710cc52ed9ab205232f47f28e09c48719922d64e: Status 404 returned error can't find the container with id b31c642c300cf6ecbf8dfb42710cc52ed9ab205232f47f28e09c48719922d64e
Oct 30 20:50:56.760474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.760441    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:56.760840 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.760512    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:56.760840 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.760539    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:50:56.768315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.768230    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"
Oct 30 20:50:56.768315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.768254    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7" event={"ID":"0657c174-450e-41e8-b3f5-9b7fe0474d18","Type":"ContainerDied","Data":"2f919f09b54288452cd7e3de4bd72c5dd8d6bee4a6af95956ad13be2e018715f"}
Oct 30 20:50:56.768315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.768311    4852 scope.go:122] "RemoveContainer" containerID="fb2fb0f9f11b13229770a7c6522df9c7a849da1b97b5692753d17ed0059f1856"
Oct 30 20:50:56.771011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.770792    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" event={"ID":"df04b00d-07bc-453d-a7fa-2a8622fb5a0c","Type":"ContainerStarted","Data":"b31c642c300cf6ecbf8dfb42710cc52ed9ab205232f47f28e09c48719922d64e"}
Oct 30 20:50:56.772861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.772841    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"
Oct 30 20:50:56.772977 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.772860    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs" event={"ID":"ec91dca4-c220-48e7-af17-a5c1780eabda","Type":"ContainerDied","Data":"a13569e6871fcc07c3d7350c436752f9c7ee52c309b88d0ac444e280da460d4b"}
Oct 30 20:50:56.772977 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.772907    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a13569e6871fcc07c3d7350c436752f9c7ee52c309b88d0ac444e280da460d4b"
Oct 30 20:50:56.819489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.819466    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"]
Oct 30 20:50:56.827867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:56.827840    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/e2e-conflict-a-b39ef5b3-kserve-66476cc84b-xgmd7"]
Oct 30 20:50:57.786334 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:57.786285    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" event={"ID":"df04b00d-07bc-453d-a7fa-2a8622fb5a0c","Type":"ContainerStarted","Data":"da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa"}
Oct 30 20:50:57.788289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:57.788266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:57.810551 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:57.810492    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podStartSLOduration=1.810472353 podStartE2EDuration="1.810472353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:50:57.808938337 +0000 UTC m=+765.923672539" watchObservedRunningTime="2026-10-30 20:50:57.810472353 +0000 UTC m=+765.925206544"
Oct 30 20:50:58.004606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:58.004571    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0657c174-450e-41e8-b3f5-9b7fe0474d18" path="/var/lib/kubelet/pods/0657c174-450e-41e8-b3f5-9b7fe0474d18/volumes"
Oct 30 20:50:58.471412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:58.470440    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:50:58.476410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:58.476334    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:50:58.476601 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:58.476420    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 20:50:58.707130 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:58.707100    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:50:59.471412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:59.471096    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:50:59.471412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:59.471152    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 20:50:59.703207 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:50:59.703032    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:00.456955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:00.456087    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:51:00.456955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:00.456149    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 20:51:00.958179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:00.958130    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:01.468411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:01.468090    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:51:01.468411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:01.468148    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 20:51:02.345441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.345406    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"]
Oct 30 20:51:02.425950 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.425907    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:02.442292 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.442131    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"]
Oct 30 20:51:02.443218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.442908    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.536550 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.536511    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:51:02.537507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.537464    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 20:51:02.641220 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641112    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641220 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641177    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641282    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641309    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgk9w\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-kube-api-access-cgk9w\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641432    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641469    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641580    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641624    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.641741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.641668    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1dd97b-0870-4c4d-b313-685d43285e96-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743025 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.742988    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743027    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgk9w\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-kube-api-access-cgk9w\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743085    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743112    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743190    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743492 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743231    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743492 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743267    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1dd97b-0870-4c4d-b313-685d43285e96-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743491    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-data\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743549    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743595    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.743992 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.743965    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-certs\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.744098 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.744009    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.744181 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.744160    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1dd97b-0870-4c4d-b313-685d43285e96-istiod-ca-cert\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.744234 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.744191    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-credential-socket\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.746890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.746861    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-envoy\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.748936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.748905    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-podinfo\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.782940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.782904    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-token\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.785154 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.785121    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgk9w\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-kube-api-access-cgk9w\") pod \"e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") " pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:02.920345 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:02.920311    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:03.229297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.229269    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"]
Oct 30 20:51:03.231552 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:51:03.231526    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1dd97b_0870_4c4d_b313_685d43285e96.slice/crio-09360bcbaffa755929527a5b5e0f7f5e7088fa1b41eccaea9fa72b751c89833b WatchSource:0}: Error finding container 09360bcbaffa755929527a5b5e0f7f5e7088fa1b41eccaea9fa72b751c89833b: Status 404 returned error can't find the container with id 09360bcbaffa755929527a5b5e0f7f5e7088fa1b41eccaea9fa72b751c89833b
Oct 30 20:51:03.234035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.234005    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:51:03.234111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.234075    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:51:03.234111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.234106    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:51:03.332022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.331982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:03.455075 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.455025    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 20:51:03.455655 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.455094    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 20:51:03.566248 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.566206    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:03.988628 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.988535    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" event={"ID":"cd1dd97b-0870-4c4d-b313-685d43285e96","Type":"ContainerStarted","Data":"35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7"}
Oct 30 20:51:03.988628 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:03.988606    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" event={"ID":"cd1dd97b-0870-4c4d-b313-685d43285e96","Type":"ContainerStarted","Data":"09360bcbaffa755929527a5b5e0f7f5e7088fa1b41eccaea9fa72b751c89833b"}
Oct 30 20:51:04.020305 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:04.020250    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podStartSLOduration=3.020233074 podStartE2EDuration="3.020233074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:04.017078967 +0000 UTC m=+772.131813193" watchObservedRunningTime="2026-10-30 20:51:04.020233074 +0000 UTC m=+772.134967265"
Oct 30 20:51:04.702887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:04.702846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:04.921421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:04.921363    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:04.923553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:04.923525    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 20:51:04.923629 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:04.923580    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 20:51:05.199723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:05.199689    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:51:05.200339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:05.200305    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:51:05.201539 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:05.201514    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 20:51:05.903544 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:05.903500    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:05.921103 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:05.921067    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 20:51:05.921275 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:05.921122    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 20:51:06.798865 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:06.798827    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:06.921348 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:06.921310    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 20:51:06.921775 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:06.921376    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 20:51:07.702739 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:07.702703    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:07.921221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:07.921187    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 20:51:07.921420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:07.921243    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 20:51:08.645736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.645699    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:08.837109 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.837077    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"]
Oct 30 20:51:08.841284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.841262    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.845591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.845564    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:51:08.846158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.846132    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:51:08.853413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.849963    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"]
Oct 30 20:51:08.885339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.885293    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.885620 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.885594    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9hbc\" (UniqueName: \"kubernetes.io/projected/85006acd-6c86-433a-a92f-2eb6770d7373-kube-api-access-z9hbc\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.885865 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.885846    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.921269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.921198    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 20:51:08.921269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.921243    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 20:51:08.987713 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.987675    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.988094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.987813    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.988094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.987845    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hbc\" (UniqueName: \"kubernetes.io/projected/85006acd-6c86-433a-a92f-2eb6770d7373-kube-api-access-z9hbc\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.988367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.988345    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.990149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.990128    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:08.996499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:08.996481    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hbc\" (UniqueName: \"kubernetes.io/projected/85006acd-6c86-433a-a92f-2eb6770d7373-kube-api-access-z9hbc\") pod \"maas-api-revoke-sub-7d03c6c9-ppd54\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:09.188572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.188491    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:09.201525 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.201502    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"]
Oct 30 20:51:09.206854 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.206796    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"]
Oct 30 20:51:09.207055 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.206910    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295150    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295193    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295279    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295310    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb76p\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-kube-api-access-qb76p\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295327    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295341    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295358    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5365800-223d-4943-8936-0fbf3b641703-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295494    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.295774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.295532    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5365800-223d-4943-8936-0fbf3b641703-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.396829 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.396779    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb76p\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-kube-api-access-qb76p\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397029 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.396842    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397029 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.396873    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397029 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.396904    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5365800-223d-4943-8936-0fbf3b641703-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.397057    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.397096    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5365800-223d-4943-8936-0fbf3b641703-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397644 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.397619    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.397681    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397860 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.397842    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.397927 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.397865    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.398075 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.398036    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.400926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.398294    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.400926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.398381    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.400926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.400101    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5365800-223d-4943-8936-0fbf3b641703-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.406024 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.405999    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.406858 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.406828    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5365800-223d-4943-8936-0fbf3b641703-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.407337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.407312    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb76p\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-kube-api-access-qb76p\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.413091 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.413066    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.500955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.500925    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"]
Oct 30 20:51:09.501984 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:51:09.501961    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85006acd_6c86_433a_a92f_2eb6770d7373.slice/crio-8590896f6790d9922b58551e3a2b263df87ab97f8d97021b651e1f79fe3ce32f WatchSource:0}: Error finding container 8590896f6790d9922b58551e3a2b263df87ab97f8d97021b651e1f79fe3ce32f: Status 404 returned error can't find the container with id 8590896f6790d9922b58551e3a2b263df87ab97f8d97021b651e1f79fe3ce32f
Oct 30 20:51:09.567436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.567380    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:09.740268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.740228    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:09.861938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.861908    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"]
Oct 30 20:51:09.864349 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:51:09.864321    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5365800_223d_4943_8936_0fbf3b641703.slice/crio-d21d818ef308e34c9c24925cd12a59a24555c8773d5f4b0294f991a55ede2bd5 WatchSource:0}: Error finding container d21d818ef308e34c9c24925cd12a59a24555c8773d5f4b0294f991a55ede2bd5: Status 404 returned error can't find the container with id d21d818ef308e34c9c24925cd12a59a24555c8773d5f4b0294f991a55ede2bd5
Oct 30 20:51:09.867491 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.867460    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:51:09.867576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.867527    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:51:09.867576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.867556    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:51:09.922060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.922025    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 20:51:09.922206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:09.922093    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 20:51:10.036063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.035957    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" event={"ID":"f5365800-223d-4943-8936-0fbf3b641703","Type":"ContainerStarted","Data":"77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149"}
Oct 30 20:51:10.036063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.036030    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" event={"ID":"f5365800-223d-4943-8936-0fbf3b641703","Type":"ContainerStarted","Data":"d21d818ef308e34c9c24925cd12a59a24555c8773d5f4b0294f991a55ede2bd5"}
Oct 30 20:51:10.038080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.038055    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54" event={"ID":"85006acd-6c86-433a-a92f-2eb6770d7373","Type":"ContainerStarted","Data":"ad3a026679ad077e4ee9250e221e065e87334bf28fad9b23518f43e08fb38303"}
Oct 30 20:51:10.038216 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.038096    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54" event={"ID":"85006acd-6c86-433a-a92f-2eb6770d7373","Type":"ContainerStarted","Data":"8590896f6790d9922b58551e3a2b263df87ab97f8d97021b651e1f79fe3ce32f"}
Oct 30 20:51:10.063762 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.063703    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podStartSLOduration=1.063682655 podStartE2EDuration="1.063682655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:10.057073054 +0000 UTC m=+778.171807245" watchObservedRunningTime="2026-10-30 20:51:10.063682655 +0000 UTC m=+778.178416848"
Oct 30 20:51:10.083475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.083428    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54" podStartSLOduration=2.083411312 podStartE2EDuration="2.083411312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:10.080993989 +0000 UTC m=+778.195728179" watchObservedRunningTime="2026-10-30 20:51:10.083411312 +0000 UTC m=+778.198145503"
Oct 30 20:51:10.200768 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.200730    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"]
Oct 30 20:51:10.215456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.211116    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.215456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.214585    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"]
Oct 30 20:51:10.217819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.217066    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"distinct-2-w0-bc102e-kserve-self-signed-certs\""
Oct 30 20:51:10.217819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.217176    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"kube-root-ca.crt\""
Oct 30 20:51:10.217819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.217397    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"openshift-service-ca.crt\""
Oct 30 20:51:10.217819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.217459    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc102e\"/\"default-dockercfg-s2c6d\""
Oct 30 20:51:10.240988 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.240954    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:10.313007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.312901    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-model-cache\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.313184 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.313013    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-dshm\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.313184 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.313115    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-home\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.313317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.313185    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-tmp-dir\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.313317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.313249    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c01a375-cdb0-48e7-8575-8054efd4ead6-tls-certs\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.313317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.313280    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmlrj\" (UniqueName: \"kubernetes.io/projected/2c01a375-cdb0-48e7-8575-8054efd4ead6-kube-api-access-tmlrj\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.414959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.414917    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-dshm\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.415112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.415032    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-home\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.415112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.415105    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-tmp-dir\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.415236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.415149    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c01a375-cdb0-48e7-8575-8054efd4ead6-tls-certs\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.415236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.415181    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmlrj\" (UniqueName: \"kubernetes.io/projected/2c01a375-cdb0-48e7-8575-8054efd4ead6-kube-api-access-tmlrj\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.415358 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.415339    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-model-cache\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.416021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.415995    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-model-cache\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.417090 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.417056    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-tmp-dir\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.417744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.417721    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-home\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.419738 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.419715    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-dshm\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.420046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.419994    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c01a375-cdb0-48e7-8575-8054efd4ead6-tls-certs\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.426629 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.426597    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmlrj\" (UniqueName: \"kubernetes.io/projected/2c01a375-cdb0-48e7-8575-8054efd4ead6-kube-api-access-tmlrj\") pod \"distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") " pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.528587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.528421    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:10.571458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.568797    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:10.571458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.570358    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 20:51:10.571458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.570428    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 20:51:10.818638 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.818602    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:10.861592 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.861425    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"]
Oct 30 20:51:10.888780 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:51:10.888750    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c01a375_cdb0_48e7_8575_8054efd4ead6.slice/crio-2d13cb580600cf7421af777981def1881550464d36ce806cf0d8fd8abd05fb14 WatchSource:0}: Error finding container 2d13cb580600cf7421af777981def1881550464d36ce806cf0d8fd8abd05fb14: Status 404 returned error can't find the container with id 2d13cb580600cf7421af777981def1881550464d36ce806cf0d8fd8abd05fb14
Oct 30 20:51:10.925767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.925739    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:10.926041 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.926005    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:10.926814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:10.926793    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 20:51:11.091867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.091822    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" event={"ID":"2c01a375-cdb0-48e7-8575-8054efd4ead6","Type":"ContainerStarted","Data":"b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f"}
Oct 30 20:51:11.092291 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.091896    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" event={"ID":"2c01a375-cdb0-48e7-8575-8054efd4ead6","Type":"ContainerStarted","Data":"2d13cb580600cf7421af777981def1881550464d36ce806cf0d8fd8abd05fb14"}
Oct 30 20:51:11.094227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.094198    4852 generic.go:356] "Generic (PLEG): container finished" podID="85006acd-6c86-433a-a92f-2eb6770d7373" containerID="ad3a026679ad077e4ee9250e221e065e87334bf28fad9b23518f43e08fb38303" exitCode=0
Oct 30 20:51:11.094429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.094288    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54" event={"ID":"85006acd-6c86-433a-a92f-2eb6770d7373","Type":"ContainerDied","Data":"ad3a026679ad077e4ee9250e221e065e87334bf28fad9b23518f43e08fb38303"}
Oct 30 20:51:11.114548 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.114453    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podStartSLOduration=1.114435762 podStartE2EDuration="1.114435762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:11.110795964 +0000 UTC m=+779.225530150" watchObservedRunningTime="2026-10-30 20:51:11.114435762 +0000 UTC m=+779.229169954"
Oct 30 20:51:11.528978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.528944    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:11.572414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.571958    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 20:51:11.572414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:11.572041    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 20:51:12.570273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:12.569821    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 20:51:12.570273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:12.569881    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 20:51:12.690848 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:12.684826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:12.893411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:12.888847    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:13.047455 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.047416    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-openshift-ca\") pod \"85006acd-6c86-433a-a92f-2eb6770d7373\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") "
Oct 30 20:51:13.070580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.070539    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-xks-ca\") pod \"85006acd-6c86-433a-a92f-2eb6770d7373\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") "
Oct 30 20:51:13.071025 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.071000    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85006acd-6c86-433a-a92f-2eb6770d7373-kube-api-access-z9hbc\" (UniqueName: \"kubernetes.io/projected/85006acd-6c86-433a-a92f-2eb6770d7373-kube-api-access-z9hbc\") pod \"85006acd-6c86-433a-a92f-2eb6770d7373\" (UID: \"85006acd-6c86-433a-a92f-2eb6770d7373\") "
Oct 30 20:51:13.105577 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.105542    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-openshift-ca" pod "85006acd-6c86-433a-a92f-2eb6770d7373" (UID: "85006acd-6c86-433a-a92f-2eb6770d7373"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:51:13.133473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.128688    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85006acd-6c86-433a-a92f-2eb6770d7373-kube-api-access-z9hbc" pod "85006acd-6c86-433a-a92f-2eb6770d7373" (UID: "85006acd-6c86-433a-a92f-2eb6770d7373"). InnerVolumeSpecName "kube-api-access-z9hbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:51:13.196414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.191127    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:13.196414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.191256    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9hbc\" (UniqueName: \"kubernetes.io/projected/85006acd-6c86-433a-a92f-2eb6770d7373-kube-api-access-z9hbc\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:13.280412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.280349    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-xks-ca" pod "85006acd-6c86-433a-a92f-2eb6770d7373" (UID: "85006acd-6c86-433a-a92f-2eb6770d7373"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:51:13.336427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.316299    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/85006acd-6c86-433a-a92f-2eb6770d7373-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:13.362875 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.362766    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"
Oct 30 20:51:13.382439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.380422    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54" event={"ID":"85006acd-6c86-433a-a92f-2eb6770d7373","Type":"ContainerDied","Data":"8590896f6790d9922b58551e3a2b263df87ab97f8d97021b651e1f79fe3ce32f"}
Oct 30 20:51:13.382439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.380503    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8590896f6790d9922b58551e3a2b263df87ab97f8d97021b651e1f79fe3ce32f"
Oct 30 20:51:13.586408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.567926    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 20:51:13.586408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.568005    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 20:51:13.636474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:13.635775    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:14.152493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:14.150826    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:14.280922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:14.280866    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:14.471562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:14.471461    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:14.570908 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:14.570868    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 20:51:14.571246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:14.571193    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 20:51:15.272413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:15.269274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:15.573930 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:15.573826    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 20:51:15.574590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:15.574546    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 20:51:16.273755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:16.273714    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:16.568339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:16.568251    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 20:51:16.568494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:16.568338    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 20:51:17.157307 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:17.157271    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:17.905057 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:17.905021    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:17.905490 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:17.905325    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:17.906151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:17.906130    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 20:51:17.943643 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:17.943611    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:18.792646 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:18.792600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:19.837608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:19.837549    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:20.134512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:20.134407    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:20.385588 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:20.385488    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:20.560413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:20.557000    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:20.635870 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:20.635778    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:21.138399 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.138352    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"]
Oct 30 20:51:21.138960 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.138942    4852 cpu_manager.go:414] "containerMap: removing container" podUID="85006acd-6c86-433a-a92f-2eb6770d7373" containerName="revoke-keys"
Oct 30 20:51:21.139011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.138962    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85006acd-6c86-433a-a92f-2eb6770d7373" containerName="revoke-keys"
Oct 30 20:51:21.139058 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.139049    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="85006acd-6c86-433a-a92f-2eb6770d7373" containerName="revoke-keys"
Oct 30 20:51:21.142193 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.142177    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.152943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.152919    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:51:21.153062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.152983    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:51:21.156940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.156915    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"]
Oct 30 20:51:21.190611 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.190572    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:21.197793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.197760    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.197966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.197907    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj66l\" (UniqueName: \"kubernetes.io/projected/92fdd598-afde-49a4-a47c-2e4ae99fc977-kube-api-access-fj66l\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.197966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.197939    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.299534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.299495    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.299725 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.299655    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj66l\" (UniqueName: \"kubernetes.io/projected/92fdd598-afde-49a4-a47c-2e4ae99fc977-kube-api-access-fj66l\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.299725 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.299693    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.305127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.305093    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.316978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.316852    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.329546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.329513    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj66l\" (UniqueName: \"kubernetes.io/projected/92fdd598-afde-49a4-a47c-2e4ae99fc977-kube-api-access-fj66l\") pod \"maas-api-revoke-sub-d73f3b85-t2hh6\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.446300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.446216    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:21.460007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.459981    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:21.585583 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:51:21.582297    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:51:21.748989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.748960    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"]
Oct 30 20:51:21.749101 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:51:21.748995    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92fdd598_afde_49a4_a47c_2e4ae99fc977.slice/crio-3186eccc269ceb2bf373453171f3b2446b82dd2cf3266c800f4935aef5c56d54 WatchSource:0}: Error finding container 3186eccc269ceb2bf373453171f3b2446b82dd2cf3266c800f4935aef5c56d54: Status 404 returned error can't find the container with id 3186eccc269ceb2bf373453171f3b2446b82dd2cf3266c800f4935aef5c56d54
Oct 30 20:51:21.923022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:21.922991    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:22.443582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:22.443487    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6" event={"ID":"92fdd598-afde-49a4-a47c-2e4ae99fc977","Type":"ContainerStarted","Data":"e00c91d2e068eaab25a6d3f2deef5c299114c08164e487b9187239b7a7acb6f4"}
Oct 30 20:51:22.444031 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:22.443587    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6" event={"ID":"92fdd598-afde-49a4-a47c-2e4ae99fc977","Type":"ContainerStarted","Data":"3186eccc269ceb2bf373453171f3b2446b82dd2cf3266c800f4935aef5c56d54"}
Oct 30 20:51:22.459642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:22.459375    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6" podStartSLOduration=1.459358624 podStartE2EDuration="1.459358624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:22.458552477 +0000 UTC m=+790.573286677" watchObservedRunningTime="2026-10-30 20:51:22.459358624 +0000 UTC m=+790.574092815"
Oct 30 20:51:22.650332 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:22.650290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:23.379533 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:23.379499    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:23.457883 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:23.457798    4852 generic.go:356] "Generic (PLEG): container finished" podID="92fdd598-afde-49a4-a47c-2e4ae99fc977" containerID="e00c91d2e068eaab25a6d3f2deef5c299114c08164e487b9187239b7a7acb6f4" exitCode=0
Oct 30 20:51:23.457883 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:23.457860    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6" event={"ID":"92fdd598-afde-49a4-a47c-2e4ae99fc977","Type":"ContainerDied","Data":"e00c91d2e068eaab25a6d3f2deef5c299114c08164e487b9187239b7a7acb6f4"}
Oct 30 20:51:24.250643 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.250600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:24.651174 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.651133    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:24.752680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.752644    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-openshift-ca\") pod \"92fdd598-afde-49a4-a47c-2e4ae99fc977\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") "
Oct 30 20:51:24.752860 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.752686    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-xks-ca\") pod \"92fdd598-afde-49a4-a47c-2e4ae99fc977\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") "
Oct 30 20:51:24.752860 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.752731    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92fdd598-afde-49a4-a47c-2e4ae99fc977-kube-api-access-fj66l\" (UniqueName: \"kubernetes.io/projected/92fdd598-afde-49a4-a47c-2e4ae99fc977-kube-api-access-fj66l\") pod \"92fdd598-afde-49a4-a47c-2e4ae99fc977\" (UID: \"92fdd598-afde-49a4-a47c-2e4ae99fc977\") "
Oct 30 20:51:24.753091 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.753065    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-openshift-ca" pod "92fdd598-afde-49a4-a47c-2e4ae99fc977" (UID: "92fdd598-afde-49a4-a47c-2e4ae99fc977"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:51:24.753271 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.753245    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:24.754957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.754931    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-xks-ca" pod "92fdd598-afde-49a4-a47c-2e4ae99fc977" (UID: "92fdd598-afde-49a4-a47c-2e4ae99fc977"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:51:24.755171 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.755143    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92fdd598-afde-49a4-a47c-2e4ae99fc977-kube-api-access-fj66l" pod "92fdd598-afde-49a4-a47c-2e4ae99fc977" (UID: "92fdd598-afde-49a4-a47c-2e4ae99fc977"). InnerVolumeSpecName "kube-api-access-fj66l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:51:24.854447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.854334    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/92fdd598-afde-49a4-a47c-2e4ae99fc977-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:24.854447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:24.854365    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj66l\" (UniqueName: \"kubernetes.io/projected/92fdd598-afde-49a4-a47c-2e4ae99fc977-kube-api-access-fj66l\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:25.068756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:25.068718    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:25.489720 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:25.489688    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"
Oct 30 20:51:25.489894 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:25.489716    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6" event={"ID":"92fdd598-afde-49a4-a47c-2e4ae99fc977","Type":"ContainerDied","Data":"3186eccc269ceb2bf373453171f3b2446b82dd2cf3266c800f4935aef5c56d54"}
Oct 30 20:51:25.489894 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:25.489764    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3186eccc269ceb2bf373453171f3b2446b82dd2cf3266c800f4935aef5c56d54"
Oct 30 20:51:26.071521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:26.071489    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:27.048967 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.048930    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"]
Oct 30 20:51:27.049574 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.049558    4852 cpu_manager.go:414] "containerMap: removing container" podUID="92fdd598-afde-49a4-a47c-2e4ae99fc977" containerName="revoke-keys"
Oct 30 20:51:27.049656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.049575    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92fdd598-afde-49a4-a47c-2e4ae99fc977" containerName="revoke-keys"
Oct 30 20:51:27.049704 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.049693    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="92fdd598-afde-49a4-a47c-2e4ae99fc977" containerName="revoke-keys"
Oct 30 20:51:27.053629 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.053606    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.065601 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.065570    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"]
Oct 30 20:51:27.066573 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.066546    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-8e4acb\"/\"distinct-w2-8e4acb-kserve-self-signed-certs\""
Oct 30 20:51:27.083981 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.083953    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqlj2\" (UniqueName: \"kubernetes.io/projected/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-kube-api-access-bqlj2\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.084451 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.084069    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-dshm\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.084451 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.084100    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tls-certs\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.084451 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.084275    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tmp-dir\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.084451 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.084320    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-model-cache\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.084691 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.084451    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-home\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.107618 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.107576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:27.186073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186035    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tls-certs\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186224 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186167    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tmp-dir\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186224 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186199    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-model-cache\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186278    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-home\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186459 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186438    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqlj2\" (UniqueName: \"kubernetes.io/projected/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-kube-api-access-bqlj2\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186627 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186607    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-dshm\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186621    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tmp-dir\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186677    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-model-cache\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.186717 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.186701    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-home\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.188803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.188774    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-dshm\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.188803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.188794    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tls-certs\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.216788 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.216753    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqlj2\" (UniqueName: \"kubernetes.io/projected/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-kube-api-access-bqlj2\") pod \"distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.344870 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.344783    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:27.367650 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.367614    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:27.972412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:27.971908    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"]
Oct 30 20:51:28.662411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:28.661512    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:28.670409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:28.667914    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" event={"ID":"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e","Type":"ContainerStarted","Data":"87b0299e76254fe3878b01037d1fd764886f29d1f688114cc22697b164ae0f3f"}
Oct 30 20:51:28.886410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:28.882516    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:51:28.886410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:28.882582    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:29.089409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.085600    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:51:29.163413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.158490    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:29.163413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.158948    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": context deadline exceeded"
Oct 30 20:51:29.275425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.273002    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:51:29.343783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.342906    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:51:29.452417 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.451790    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:29.470490 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.469473    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:29.665478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.664492    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:29.665478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:29.665025    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:51:30.050780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.050729    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:30.101668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.101599    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:30.144414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.127427    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:51:30.164700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.162522    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:30.164700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.163425    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:51:30.275413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.274761    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:30.283549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.283510    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:51:30.359177 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.345752    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:30.359665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.359640    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:51:30.453304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.453243    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:30.459045 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.458998    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:30.482435 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.476693    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:30.482435 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.481047    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:51:30.672756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:30.672651    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:51:31.115588 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.115533    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:31.177657 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.177512    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:51:31.320412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.315481    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:51:31.352532 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.352481    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:31.455601 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.455485    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:31.503678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.503618    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:31.532780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.532720    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:31.693113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.693061    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:51:32.000723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.988855    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:51:32.000723 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:31.988927    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:32.147413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:32.123484    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:51:32.336412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:32.312509    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:32.412415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:32.410627    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:32.424412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:32.415061    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:32.638478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:32.493259    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:32.638478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:32.525889    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:51:32.724423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:32.699412    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:51:33.125575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.125513    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:51:33.285706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.285675    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:51:33.434563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.411363    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:51:33.448112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.447687    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:33.476198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.475532    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:33.500806 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.499597    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:33.527551 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.526363    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:33.551166 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.551128    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:51:33.570849 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.570809    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:51:33.616377 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.611199    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:51:33.639464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.638213    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:51:33.673440 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.668968    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"]
Oct 30 20:51:33.749983 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.749955    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:33.762532 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:33.762507    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:51:34.024172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:34.024137    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" event={"ID":"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e","Type":"ContainerStarted","Data":"cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860"}
Oct 30 20:51:34.044267 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:34.044214    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podStartSLOduration=7.044197576 podStartE2EDuration="7.044197576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:34.04308067 +0000 UTC m=+802.157814861" watchObservedRunningTime="2026-10-30 20:51:34.044197576 +0000 UTC m=+802.158931767"
Oct 30 20:51:34.308265 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:34.308171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:34.368146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:34.368113    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:34.701546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:34.701459    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:34.801635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:34.801589    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:35.097101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:35.097070    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:36.609495 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:36.609458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:37.367908 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:37.367872    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:37.380819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:37.380785    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:37.448966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:37.448925    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:38.069478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:38.069450    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:38.361217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:38.361136    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:38.669937 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:38.669847    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2" podUID="8cab6d3e-a657-4e30-89aa-eef0a39c9886" containerName="payload-processing" containerID="cri-o://c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631" gracePeriod=25
Oct 30 20:51:38.939441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:38.939351    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:51:39.032580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.032549    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-routing-overlay-volume\") pod \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") "
Oct 30 20:51:39.032580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.032582    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-kube-api-access-tszzk\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-kube-api-access-tszzk\") pod \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") "
Oct 30 20:51:39.032786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.032672    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-plugins-config-volume\") pod \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") "
Oct 30 20:51:39.032786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.032726    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-provider-credentials\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-provider-credentials\") pod \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\" (UID: \"8cab6d3e-a657-4e30-89aa-eef0a39c9886\") "
Oct 30 20:51:39.033017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.032991    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-routing-overlay-volume" pod "8cab6d3e-a657-4e30-89aa-eef0a39c9886" (UID: "8cab6d3e-a657-4e30-89aa-eef0a39c9886"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:51:39.033170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.033136    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-plugins-config-volume" pod "8cab6d3e-a657-4e30-89aa-eef0a39c9886" (UID: "8cab6d3e-a657-4e30-89aa-eef0a39c9886"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:51:39.033348 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.033326    4852 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-routing-overlay-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:39.033470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.033353    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cab6d3e-a657-4e30-89aa-eef0a39c9886-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:39.035041 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.035022    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-provider-credentials" pod "8cab6d3e-a657-4e30-89aa-eef0a39c9886" (UID: "8cab6d3e-a657-4e30-89aa-eef0a39c9886"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:51:39.035093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.035077    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-kube-api-access-tszzk" pod "8cab6d3e-a657-4e30-89aa-eef0a39c9886" (UID: "8cab6d3e-a657-4e30-89aa-eef0a39c9886"). InnerVolumeSpecName "kube-api-access-tszzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:51:39.066012 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.065981    4852 generic.go:356] "Generic (PLEG): container finished" podID="8cab6d3e-a657-4e30-89aa-eef0a39c9886" containerID="c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631" exitCode=0
Oct 30 20:51:39.066168 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.066022    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"
Oct 30 20:51:39.066168 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.066061    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2" event={"ID":"8cab6d3e-a657-4e30-89aa-eef0a39c9886","Type":"ContainerDied","Data":"c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631"}
Oct 30 20:51:39.066168 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.066117    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2" event={"ID":"8cab6d3e-a657-4e30-89aa-eef0a39c9886","Type":"ContainerDied","Data":"9366c785c06db3f34936fd850ed0aa2c7ffde9b70a21a02fd1ccb76d0b54f6a1"}
Oct 30 20:51:39.066168 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.066145    4852 scope.go:122] "RemoveContainer" containerID="c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631"
Oct 30 20:51:39.076879 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.076860    4852 scope.go:122] "RemoveContainer" containerID="c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631"
Oct 30 20:51:39.077181 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:51:39.077141    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631\": container with ID starting with c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631 not found: ID does not exist" containerID="c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631"
Oct 30 20:51:39.077181 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.077166    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631"} err="failed to get container status \"c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631\": rpc error: code = NotFound desc = could not find container \"c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631\": container with ID starting with c0848dea4eb75e9ffae8a9942b1a9c7c9104760550dd60caea36277d854a7631 not found: ID does not exist"
Oct 30 20:51:39.093268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.093242    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"]
Oct 30 20:51:39.096871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.096842    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-57db8b5885-9ncq2"]
Oct 30 20:51:39.135296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.135261    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tszzk\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-kube-api-access-tszzk\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:39.135296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.135290    4852 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8cab6d3e-a657-4e30-89aa-eef0a39c9886-provider-credentials\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:51:39.337720 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:39.337680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:40.004989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:40.004953    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cab6d3e-a657-4e30-89aa-eef0a39c9886" path="/var/lib/kubelet/pods/8cab6d3e-a657-4e30-89aa-eef0a39c9886/volumes"
Oct 30 20:51:40.142752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:40.142721    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:41.249917 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:41.249884    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:42.134271 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:42.134233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:42.830777 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:42.830737    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:43.676233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.676196    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"]
Oct 30 20:51:43.676814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.676794    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8cab6d3e-a657-4e30-89aa-eef0a39c9886" containerName="payload-processing"
Oct 30 20:51:43.676814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.676813    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cab6d3e-a657-4e30-89aa-eef0a39c9886" containerName="payload-processing"
Oct 30 20:51:43.676981 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.676919    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cab6d3e-a657-4e30-89aa-eef0a39c9886" containerName="payload-processing"
Oct 30 20:51:43.681949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.681922    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.709804 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.709735    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"]
Oct 30 20:51:43.712608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.712574    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-53aa47\"/\"distinct-w4-53aa47-kserve-self-signed-certs\""
Oct 30 20:51:43.720835 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.720804    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:43.798567 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.798528    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-home\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.798742 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.798607    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfts6\" (UniqueName: \"kubernetes.io/projected/47ff9971-1706-43a5-9123-e51ce4028da5-kube-api-access-bfts6\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.798807 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.798747    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-tmp-dir\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.798807 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.798793    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-dshm\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.798874 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.798821    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ff9971-1706-43a5-9123-e51ce4028da5-tls-certs\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.798993 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.798977    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-model-cache\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.822603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.822571    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:43.905146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.901454    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-model-cache\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.916093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.916056    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-home\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.916341 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.916318    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfts6\" (UniqueName: \"kubernetes.io/projected/47ff9971-1706-43a5-9123-e51ce4028da5-kube-api-access-bfts6\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.916590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.916567    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-tmp-dir\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.946526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.933592    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-dshm\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.946526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.933662    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ff9971-1706-43a5-9123-e51ce4028da5-tls-certs\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.946526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.917049    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-home\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.946526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.906947    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-model-cache\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.946526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.917753    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-tmp-dir\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.953789 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.953717    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-dshm\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.954182 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.954108    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ff9971-1706-43a5-9123-e51ce4028da5-tls-certs\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:43.972127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:43.972057    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfts6\" (UniqueName: \"kubernetes.io/projected/47ff9971-1706-43a5-9123-e51ce4028da5-kube-api-access-bfts6\") pod \"distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:44.022773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:44.022737    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:44.375408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:44.370668    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"]
Oct 30 20:51:45.146667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:45.146619    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" event={"ID":"47ff9971-1706-43a5-9123-e51ce4028da5","Type":"ContainerStarted","Data":"15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4"}
Oct 30 20:51:45.147259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:45.147233    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" event={"ID":"47ff9971-1706-43a5-9123-e51ce4028da5","Type":"ContainerStarted","Data":"772ffb154a62763e83b2ec8e6baba787c73cc5a84a4eca37bd0373465d6d077e"}
Oct 30 20:51:45.362151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:45.344088    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podStartSLOduration=2.34406407 podStartE2EDuration="2.34406407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:45.211781559 +0000 UTC m=+813.326515749" watchObservedRunningTime="2026-10-30 20:51:45.34406407 +0000 UTC m=+813.458798263"
Oct 30 20:51:45.628427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:45.614120    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:45.745411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:45.729902    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:45.745411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:45.730212    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:51:46.053683 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.053647    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:46.081312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.081267    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:46.129966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.129907    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:46.194736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.194681    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:46.317630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.317492    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:46.376680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.376632    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:51:46.507546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.507487    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:46.529534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.529481    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:46.742474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.732516    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:46.742474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:46.733322    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:51:47.050433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.045087    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-7pzkt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:51:47.050433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.045195    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:47.118415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.089679    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:47.134570 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.134513    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:47.219552 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.219504    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:51:47.310469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.307467    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:47.310469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.307585    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:51:47.310469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.307664    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:51:47.351349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.322874    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:51:47.351349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.323656    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:51:47.400409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.379496    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:51:47.400409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.381279    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:51:47.513192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.513146    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 20:51:47.529959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.529928    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:47.531134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.531101    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:47.532498 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.532475    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:51:47.660852 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.636667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:47.768577 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.768527    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:47.975463 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:47.972503    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"]
Oct 30 20:51:48.058444 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.029368    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:48.147671 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.147641    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.222811 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.222781    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-8e4acb\"/\"distinct-2-w2-8e4acb-kserve-self-signed-certs\""
Oct 30 20:51:48.239827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.239717    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-tmp-dir\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.242553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.242518    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-home\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.251220 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.251177    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-dshm\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.251583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.251556    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-model-cache\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.255535 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.255475    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdkdz\" (UniqueName: \"kubernetes.io/projected/1fa0a83b-ab79-4a98-8030-1b05158342d1-kube-api-access-zdkdz\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.255944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.255917    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa0a83b-ab79-4a98-8030-1b05158342d1-tls-certs\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.495441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.495324    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:48.500506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.500461    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:48.516496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.513574    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:48.516496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.513882    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:51:48.524706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.523430    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:51:48.524706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.523720    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:51:48.539612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.539567    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 20:51:48.540187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.540150    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:51:48.584116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.583985    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-b8h86 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:51:48.584116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.584045    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-b8h86" podUID="d2ea773e-9dd0-4894-8610-4c5c8f465087" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:48.620416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.615532    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-dshm\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.620416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.615598    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-model-cache\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.620416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.615652    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdkdz\" (UniqueName: \"kubernetes.io/projected/1fa0a83b-ab79-4a98-8030-1b05158342d1-kube-api-access-zdkdz\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.620416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.615736    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa0a83b-ab79-4a98-8030-1b05158342d1-tls-certs\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.620416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.615880    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-tmp-dir\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.620416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.615915    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-home\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.620416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.616486    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-home\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.755570 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.754256    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-model-cache\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.755732 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.755593    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-6nj94 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:51:48.755732 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.755645    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:48.772751 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.772078    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa0a83b-ab79-4a98-8030-1b05158342d1-tls-certs\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.911297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.910688    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:51:48.938195 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.938135    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:51:48.967394 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.967340    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"]
Oct 30 20:51:48.975321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.975183    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-tmp-dir\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:48.992437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:48.992374    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-dshm\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:49.054072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.053977    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:51:49.054365 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.054347    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:51:49.054542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.054529    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:51:49.054719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.054697    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:51:49.054864 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.054851    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:51:49.055017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.055004    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:51:49.159828 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.152789    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdkdz\" (UniqueName: \"kubernetes.io/projected/1fa0a83b-ab79-4a98-8030-1b05158342d1-kube-api-access-zdkdz\") pod \"distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") " pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:49.399464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.374901    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:49.479588 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:49.465045    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:51:50.072062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:50.071933    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"]
Oct 30 20:51:50.074408 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:51:50.074189    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fa0a83b_ab79_4a98_8030_1b05158342d1.slice/crio-f25315ff07f3cdf829ba4b0508711c73d273ad22b34c23959397a0f4243e4898 WatchSource:0}: Error finding container f25315ff07f3cdf829ba4b0508711c73d273ad22b34c23959397a0f4243e4898: Status 404 returned error can't find the container with id f25315ff07f3cdf829ba4b0508711c73d273ad22b34c23959397a0f4243e4898
Oct 30 20:51:50.257560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:50.257515    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:50.768770 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:50.768732    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" event={"ID":"1fa0a83b-ab79-4a98-8030-1b05158342d1","Type":"ContainerStarted","Data":"2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97"}
Oct 30 20:51:50.768770 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:50.768783    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" event={"ID":"1fa0a83b-ab79-4a98-8030-1b05158342d1","Type":"ContainerStarted","Data":"f25315ff07f3cdf829ba4b0508711c73d273ad22b34c23959397a0f4243e4898"}
Oct 30 20:51:50.785287 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:50.785228    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podStartSLOduration=3.785210494 podStartE2EDuration="3.785210494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:51:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:51:50.784314113 +0000 UTC m=+818.899048304" watchObservedRunningTime="2026-10-30 20:51:50.785210494 +0000 UTC m=+818.899944686"
Oct 30 20:51:51.238536 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:51.238453    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:51.394399 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:51.394361    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:51.558776 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:51:51.558741    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:51:52.237398 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:52.237350    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:53.226984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:53.226954    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:54.022846 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:54.022817    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:54.033087 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:54.033058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:54.036115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:54.036091    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:54.808996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:54.808961    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:51:55.258048 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:55.258013    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:56.262062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:56.262015    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:57.137046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:57.136989    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:58.049053 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:58.049013    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:58.852655 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:58.852602    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:59.394445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:59.394378    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:59.408817 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:59.408781    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:51:59.700944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:59.700838    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:51:59.850735 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:51:59.850700    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:52:00.630334 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:00.630295    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:01.543830 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:01.543788    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:02.466951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:02.466893    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:03.356164 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.356123    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:03.440999 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.440965    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"]
Oct 30 20:52:03.447056 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.447031    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.452774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.452753    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-53aa47\"/\"distinct-2-w4-53aa47-kserve-self-signed-certs\""
Oct 30 20:52:03.456898 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.456870    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"]
Oct 30 20:52:03.482053 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.482012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:03.491955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.491925    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/144803a1-e592-4af8-bd4d-c4e33ddf401a-kube-api-access-7g2f2\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.492125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.491963    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144803a1-e592-4af8-bd4d-c4e33ddf401a-tls-certs\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.492199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.492140    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-dshm\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.492199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.492183    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-model-cache\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.492329 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.492313    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-home\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.492422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.492351    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-tmp-dir\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.584633 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.584600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:03.623367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.622909    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-home\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.623367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.622977    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-tmp-dir\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.623367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.623118    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/144803a1-e592-4af8-bd4d-c4e33ddf401a-kube-api-access-7g2f2\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.623367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.623152    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144803a1-e592-4af8-bd4d-c4e33ddf401a-tls-certs\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.623367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.623291    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-dshm\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.623367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.623321    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-model-cache\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.623947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.623910    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-model-cache\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.624194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.624175    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-home\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.624445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.624424    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-tmp-dir\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.635025 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.633654    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-dshm\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.656276 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.656244    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144803a1-e592-4af8-bd4d-c4e33ddf401a-tls-certs\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.659159 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.658099    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/144803a1-e592-4af8-bd4d-c4e33ddf401a-kube-api-access-7g2f2\") pod \"distinct-2-w4-53aa47-kserve-977b56845-4gdp6\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") " pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:03.765039 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:03.765000    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:04.050338 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:04.050307    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"]
Oct 30 20:52:04.050524 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:04.050350    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod144803a1_e592_4af8_bd4d_c4e33ddf401a.slice/crio-784c38e1a16b0804ca1f1e3db1e4d67329735e0f61dac9f3c56db4f85eccf14e WatchSource:0}: Error finding container 784c38e1a16b0804ca1f1e3db1e4d67329735e0f61dac9f3c56db4f85eccf14e: Status 404 returned error can't find the container with id 784c38e1a16b0804ca1f1e3db1e4d67329735e0f61dac9f3c56db4f85eccf14e
Oct 30 20:52:04.597046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:04.597011    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:04.875609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:04.875523    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" event={"ID":"144803a1-e592-4af8-bd4d-c4e33ddf401a","Type":"ContainerStarted","Data":"ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f"}
Oct 30 20:52:04.875609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:04.875575    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" event={"ID":"144803a1-e592-4af8-bd4d-c4e33ddf401a","Type":"ContainerStarted","Data":"784c38e1a16b0804ca1f1e3db1e4d67329735e0f61dac9f3c56db4f85eccf14e"}
Oct 30 20:52:04.893701 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:04.893648    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podStartSLOduration=1.8936331389999999 podStartE2EDuration="1.893633139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:04.8930259 +0000 UTC m=+833.007760091" watchObservedRunningTime="2026-10-30 20:52:04.893633139 +0000 UTC m=+833.008367328"
Oct 30 20:52:05.628925 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:05.628891    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:05.766153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:05.766120    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:06.446889 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:06.446856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:07.070148 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.070108    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"]
Oct 30 20:52:07.074301 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.074283    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.078413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.078357    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:52:07.078540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.078458    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:52:07.082157 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.082131    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"]
Oct 30 20:52:07.181772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.181741    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.181881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.181851    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjwt6\" (UniqueName: \"kubernetes.io/projected/2305a0ae-5164-4316-9d9e-c5419e2c1135-kube-api-access-sjwt6\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.182019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.182003    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.283207 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.283160    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.283438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.283255    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.283438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.283301    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjwt6\" (UniqueName: \"kubernetes.io/projected/2305a0ae-5164-4316-9d9e-c5419e2c1135-kube-api-access-sjwt6\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.283849 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.283826    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.285631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.285606    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.291840 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.291817    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjwt6\" (UniqueName: \"kubernetes.io/projected/2305a0ae-5164-4316-9d9e-c5419e2c1135-kube-api-access-sjwt6\") pod \"maas-api-revoke-sub-66537acb-slk2p\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.354416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.354311    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:07.415606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.415572    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:07.718627 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.718600    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"]
Oct 30 20:52:07.720508 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:07.720480    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2305a0ae_5164_4316_9d9e_c5419e2c1135.slice/crio-2e61971e81fc30b00eabfbf0744a893961c7e463c79dcb51eac68a51ec01cbf8 WatchSource:0}: Error finding container 2e61971e81fc30b00eabfbf0744a893961c7e463c79dcb51eac68a51ec01cbf8: Status 404 returned error can't find the container with id 2e61971e81fc30b00eabfbf0744a893961c7e463c79dcb51eac68a51ec01cbf8
Oct 30 20:52:07.898314 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:07.898276    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p" event={"ID":"2305a0ae-5164-4316-9d9e-c5419e2c1135","Type":"ContainerStarted","Data":"2e61971e81fc30b00eabfbf0744a893961c7e463c79dcb51eac68a51ec01cbf8"}
Oct 30 20:52:08.409305 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:08.409273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:08.906455 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:08.906416    4852 generic.go:356] "Generic (PLEG): container finished" podID="2305a0ae-5164-4316-9d9e-c5419e2c1135" containerID="84bdbe70be4bf0299dcb7fcf04610bd62b35e757f2f3e88c667ea20a55bc7f4a" exitCode=0
Oct 30 20:52:08.906634 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:08.906458    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p" event={"ID":"2305a0ae-5164-4316-9d9e-c5419e2c1135","Type":"ContainerDied","Data":"84bdbe70be4bf0299dcb7fcf04610bd62b35e757f2f3e88c667ea20a55bc7f4a"}
Oct 30 20:52:09.232259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:09.232170    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:10.074774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.074746    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:10.126688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.126650    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:10.220882 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.220781    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-openshift-ca\") pod \"2305a0ae-5164-4316-9d9e-c5419e2c1135\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") "
Oct 30 20:52:10.221065 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.220894    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2305a0ae-5164-4316-9d9e-c5419e2c1135-kube-api-access-sjwt6\" (UniqueName: \"kubernetes.io/projected/2305a0ae-5164-4316-9d9e-c5419e2c1135-kube-api-access-sjwt6\") pod \"2305a0ae-5164-4316-9d9e-c5419e2c1135\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") "
Oct 30 20:52:10.221142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.221074    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-xks-ca\") pod \"2305a0ae-5164-4316-9d9e-c5419e2c1135\" (UID: \"2305a0ae-5164-4316-9d9e-c5419e2c1135\") "
Oct 30 20:52:10.221249 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.221225    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-openshift-ca" pod "2305a0ae-5164-4316-9d9e-c5419e2c1135" (UID: "2305a0ae-5164-4316-9d9e-c5419e2c1135"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:52:10.221581 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.221560    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:10.223025 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.223003    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2305a0ae-5164-4316-9d9e-c5419e2c1135-kube-api-access-sjwt6" pod "2305a0ae-5164-4316-9d9e-c5419e2c1135" (UID: "2305a0ae-5164-4316-9d9e-c5419e2c1135"). InnerVolumeSpecName "kube-api-access-sjwt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:52:10.223086 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.223046    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-xks-ca" pod "2305a0ae-5164-4316-9d9e-c5419e2c1135" (UID: "2305a0ae-5164-4316-9d9e-c5419e2c1135"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:52:10.322558 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.322525    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2305a0ae-5164-4316-9d9e-c5419e2c1135-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:10.322558 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.322559    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjwt6\" (UniqueName: \"kubernetes.io/projected/2305a0ae-5164-4316-9d9e-c5419e2c1135-kube-api-access-sjwt6\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:10.922158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.922124    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p" event={"ID":"2305a0ae-5164-4316-9d9e-c5419e2c1135","Type":"ContainerDied","Data":"2e61971e81fc30b00eabfbf0744a893961c7e463c79dcb51eac68a51ec01cbf8"}
Oct 30 20:52:10.922158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.922144    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"
Oct 30 20:52:10.922427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:10.922182    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e61971e81fc30b00eabfbf0744a893961c7e463c79dcb51eac68a51ec01cbf8"
Oct 30 20:52:11.770413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:11.769690    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:12.508411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:12.505513    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:12.548423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:12.539559    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:52:12.816506 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:12.814821    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:12.892410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:12.892123    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 20:52:12.948663 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:12.944723    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 20:52:13.004564 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.004510    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:13.039411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.038677    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:52:13.061424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.039934    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 20:52:13.497702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.497651    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:13.529121 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.528305    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": context deadline exceeded"
Oct 30 20:52:13.542040 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.541652    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:13.545680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.543517    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:52:13.555776 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.548465    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:52:13.558798 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.558767    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:52:13.576914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.576872    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:13.849412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.845926    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:13.862420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.861302    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:52:13.963753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:13.963715    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:14.099469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.099225    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:14.120424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.100475    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 20:52:14.121365 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.121334    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:52:14.121600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.121573    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:14.130890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.128145    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:14.130890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.128799    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:14.130890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.129089    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:14.339700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.339655    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:52:14.339904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.339737    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:52:14.339904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.339777    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:52:14.339904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.339813    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:52:14.339904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.339847    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:52:14.505457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.505052    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:14.505457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.505199    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:52:14.505679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.505652    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:52:14.544618 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.544575    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:52:14.552560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.552511    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:52:14.606772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.596164    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:52:14.606772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.596232    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:52:14.606772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.596992    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:52:14.645170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.645141    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:52:14.650489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.650464    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:52:14.712413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.710797    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:14.726951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.721035    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:52:14.726951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.723355    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:52:14.732268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.732241    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:52:14.735417 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.734195    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:52:14.781126 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:14.781046    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:15.199927 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.199849    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"]
Oct 30 20:52:15.200521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.200502    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2305a0ae-5164-4316-9d9e-c5419e2c1135" containerName="revoke-keys"
Oct 30 20:52:15.200521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.200521    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2305a0ae-5164-4316-9d9e-c5419e2c1135" containerName="revoke-keys"
Oct 30 20:52:15.200622 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.200616    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2305a0ae-5164-4316-9d9e-c5419e2c1135" containerName="revoke-keys"
Oct 30 20:52:15.205528 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.205511    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.217443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.217401    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:52:15.220558 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.220534    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:52:15.222184 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.222159    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"]
Oct 30 20:52:15.326980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.326628    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.326980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.326827    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rbm8\" (UniqueName: \"kubernetes.io/projected/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-kube-api-access-9rbm8\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.327289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.327257    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.428939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.428896    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.429112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.429004    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rbm8\" (UniqueName: \"kubernetes.io/projected/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-kube-api-access-9rbm8\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.429112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.429106    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.429646 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.429618    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.431439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.431401    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.437782 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.437754    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rbm8\" (UniqueName: \"kubernetes.io/projected/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-kube-api-access-9rbm8\") pod \"maas-api-revoke-sub-bc4c14da-cbhn7\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.524504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.524473    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:15.810144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:15.810113    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"]
Oct 30 20:52:15.810743 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:15.810712    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08971e8f_6005_4e56_b9e1_32deb5d3c2d6.slice/crio-2870cf1b779aa80b2f4f7121d940590309baf15ffd5ccd568acd4ae8dafb4e39 WatchSource:0}: Error finding container 2870cf1b779aa80b2f4f7121d940590309baf15ffd5ccd568acd4ae8dafb4e39: Status 404 returned error can't find the container with id 2870cf1b779aa80b2f4f7121d940590309baf15ffd5ccd568acd4ae8dafb4e39
Oct 30 20:52:16.154856 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:16.154817    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7" event={"ID":"08971e8f-6005-4e56-b9e1-32deb5d3c2d6","Type":"ContainerStarted","Data":"4b71fa914afe438d83ebacec1d0ac8e7e1016e2a0ddb65131e6ce61f060584c9"}
Oct 30 20:52:16.155150 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:16.154880    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7" event={"ID":"08971e8f-6005-4e56-b9e1-32deb5d3c2d6","Type":"ContainerStarted","Data":"2870cf1b779aa80b2f4f7121d940590309baf15ffd5ccd568acd4ae8dafb4e39"}
Oct 30 20:52:16.173652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:16.173597    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7" podStartSLOduration=1.173580972 podStartE2EDuration="1.173580972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:16.17324284 +0000 UTC m=+844.287977107" watchObservedRunningTime="2026-10-30 20:52:16.173580972 +0000 UTC m=+844.288315164"
Oct 30 20:52:16.209697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:16.209652    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:17.056324 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:17.056284    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:17.165092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:17.165058    4852 generic.go:356] "Generic (PLEG): container finished" podID="08971e8f-6005-4e56-b9e1-32deb5d3c2d6" containerID="4b71fa914afe438d83ebacec1d0ac8e7e1016e2a0ddb65131e6ce61f060584c9" exitCode=0
Oct 30 20:52:17.165296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:17.165144    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7" event={"ID":"08971e8f-6005-4e56-b9e1-32deb5d3c2d6","Type":"ContainerDied","Data":"4b71fa914afe438d83ebacec1d0ac8e7e1016e2a0ddb65131e6ce61f060584c9"}
Oct 30 20:52:17.992077 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:17.992026    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:18.333089 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.333061    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:18.370993 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.370959    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-openshift-ca\") pod \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") "
Oct 30 20:52:18.371173 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.371104    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-kube-api-access-9rbm8\" (UniqueName: \"kubernetes.io/projected/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-kube-api-access-9rbm8\") pod \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") "
Oct 30 20:52:18.371173 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.371156    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-xks-ca\") pod \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\" (UID: \"08971e8f-6005-4e56-b9e1-32deb5d3c2d6\") "
Oct 30 20:52:18.371337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.371310    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-openshift-ca" pod "08971e8f-6005-4e56-b9e1-32deb5d3c2d6" (UID: "08971e8f-6005-4e56-b9e1-32deb5d3c2d6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:52:18.371784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.371758    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:18.373235 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.373208    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-xks-ca" pod "08971e8f-6005-4e56-b9e1-32deb5d3c2d6" (UID: "08971e8f-6005-4e56-b9e1-32deb5d3c2d6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:52:18.373354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.373302    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-kube-api-access-9rbm8" pod "08971e8f-6005-4e56-b9e1-32deb5d3c2d6" (UID: "08971e8f-6005-4e56-b9e1-32deb5d3c2d6"). InnerVolumeSpecName "kube-api-access-9rbm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:52:18.473094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.473056    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rbm8\" (UniqueName: \"kubernetes.io/projected/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-kube-api-access-9rbm8\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:18.473094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.473093    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/08971e8f-6005-4e56-b9e1-32deb5d3c2d6-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:18.961152 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:18.961110    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:19.183244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:19.183201    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7" event={"ID":"08971e8f-6005-4e56-b9e1-32deb5d3c2d6","Type":"ContainerDied","Data":"2870cf1b779aa80b2f4f7121d940590309baf15ffd5ccd568acd4ae8dafb4e39"}
Oct 30 20:52:19.183616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:19.183264    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2870cf1b779aa80b2f4f7121d940590309baf15ffd5ccd568acd4ae8dafb4e39"
Oct 30 20:52:19.183616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:19.183227    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"
Oct 30 20:52:20.003560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:20.003524    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:20.794736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:20.794708    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:21.094650 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.094568    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:21.254094 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.254060    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"]
Oct 30 20:52:21.254855 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.254831    4852 cpu_manager.go:414] "containerMap: removing container" podUID="08971e8f-6005-4e56-b9e1-32deb5d3c2d6" containerName="revoke-keys"
Oct 30 20:52:21.254855 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.254855    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08971e8f-6005-4e56-b9e1-32deb5d3c2d6" containerName="revoke-keys"
Oct 30 20:52:21.255052 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.255007    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="08971e8f-6005-4e56-b9e1-32deb5d3c2d6" containerName="revoke-keys"
Oct 30 20:52:21.259084 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.259061    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.272068 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.272044    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a52bd2a1-openshift-default-dockercfg-hk4g6\""
Oct 30 20:52:21.282701 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.282657    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"]
Oct 30 20:52:21.307208 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307176    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-certs\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307226    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk54z\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-kube-api-access-mk54z\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307254    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-envoy\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307568 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307446    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-podinfo\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307568 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307539    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istiod-ca-cert\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307685 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307608    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-socket\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307709    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-credential-socket\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307730    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-data\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.307851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.307749    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-token\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.339355 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.339322    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:21.408841 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.408761    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-envoy\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.408841 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.408840    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-podinfo\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.408891    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istiod-ca-cert\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.408956    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-socket\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409036    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-credential-socket\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409060    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-data\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409085    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-token\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409119    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-certs\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409168    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk54z\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-kube-api-access-mk54z\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409453 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409425    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-socket\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409505    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-credential-socket\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409622 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409595    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-data\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409643    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-certs\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.409893 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.409876    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istiod-ca-cert\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.411174 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.411142    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-envoy\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.411414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.411382    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-podinfo\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.422228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.422201    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-token\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.422505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.422294    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk54z\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-kube-api-access-mk54z\") pod \"e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") " pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.475410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.453054    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:21.574970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.574655    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:21.580449 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:52:21.580270    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:52:21.882883 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:21.882840    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061e7d86_7f1c_40db_8ecd_76ce35bcadee.slice/crio-92cd3c8881d1dda98a40319bc7dfb20811ed4ee73eeb5dff080163f286838e5b WatchSource:0}: Error finding container 92cd3c8881d1dda98a40319bc7dfb20811ed4ee73eeb5dff080163f286838e5b: Status 404 returned error can't find the container with id 92cd3c8881d1dda98a40319bc7dfb20811ed4ee73eeb5dff080163f286838e5b
Oct 30 20:52:21.883215 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.883070    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"]
Oct 30 20:52:21.885279 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.885248    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:21.885361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.885311    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:21.885361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:21.885343    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:22.207978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:22.207884    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" event={"ID":"061e7d86-7f1c-40db-8ecd-76ce35bcadee","Type":"ContainerStarted","Data":"30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4"}
Oct 30 20:52:22.207978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:22.207955    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" event={"ID":"061e7d86-7f1c-40db-8ecd-76ce35bcadee","Type":"ContainerStarted","Data":"92cd3c8881d1dda98a40319bc7dfb20811ed4ee73eeb5dff080163f286838e5b"}
Oct 30 20:52:22.242203 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:22.242149    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podStartSLOduration=1.242135492 podStartE2EDuration="1.242135492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:22.237163833 +0000 UTC m=+850.351898023" watchObservedRunningTime="2026-10-30 20:52:22.242135492 +0000 UTC m=+850.356869682"
Oct 30 20:52:22.265054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:22.265024    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:22.575351 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:22.575318    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:22.580552 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:22.580520    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:22.945594 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:22.945508    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:23.215743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:23.215657    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:23.216784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:23.216766    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:52:23.377752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:23.377704    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:23.508735 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:23.505058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:23.744952 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:23.744888    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:24.821507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:24.821450    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:24.936773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:24.936721    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:25.979677 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:25.979644    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:26.116293 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:26.116256    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:26.249914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:26.249829    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:27.421330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:27.421277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:27.537820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:27.537775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:27.707316 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:27.707224    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:28.133169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.133131    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"]
Oct 30 20:52:28.137794 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.137774    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.156839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.156811    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"]
Oct 30 20:52:28.242765 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.242731    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.242970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.242771    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.242970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.242897    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.242970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.242941    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlg7q\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-kube-api-access-qlg7q\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.243145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.243046    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.243145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.243109    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a9f3ba-4151-4074-b25e-a12adb278946-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.243250 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.243212    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.243308 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.243263    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a9f3ba-4151-4074-b25e-a12adb278946-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.243378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.243359    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345081 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345043    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345093    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345144    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345175    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlg7q\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-kube-api-access-qlg7q\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345209    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345253    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a9f3ba-4151-4074-b25e-a12adb278946-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345271    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345664 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345300    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a9f3ba-4151-4074-b25e-a12adb278946-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345664 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345356    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345664 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345567    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345661    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345826    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.345957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.345936    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.346159 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.346140    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a9f3ba-4151-4074-b25e-a12adb278946-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.347671 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.347644    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.347776 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.347758    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a9f3ba-4151-4074-b25e-a12adb278946-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.364518 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.364492    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.364833 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.364812    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlg7q\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-kube-api-access-qlg7q\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.453881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.453798    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:28.741295 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.741265    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"]
Oct 30 20:52:28.749683 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.749642    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:28.749803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.749728    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:28.749803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.749773    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:28.824988 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.824943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:28.932823 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.932790    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:28.937953 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.937529    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"]
Oct 30 20:52:28.941934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.941906    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:28.979351 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.978981    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1fac16\""
Oct 30 20:52:28.979351 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.979222    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1fac16-dockercfg-9xxdq\""
Oct 30 20:52:28.984406 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:28.984344    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"]
Oct 30 20:52:29.065415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.061381    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8scm\" (UniqueName: \"kubernetes.io/projected/d7407621-98f1-44a3-8e23-2e7cffc888ec-kube-api-access-z8scm\") pod \"payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:29.065415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.061577    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7407621-98f1-44a3-8e23-2e7cffc888ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:29.072108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.067074    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:29.163668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.163626    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8scm\" (UniqueName: \"kubernetes.io/projected/d7407621-98f1-44a3-8e23-2e7cffc888ec-kube-api-access-z8scm\") pod \"payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:29.164490 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.164451    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7407621-98f1-44a3-8e23-2e7cffc888ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:29.169022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.168856    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7407621-98f1-44a3-8e23-2e7cffc888ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:29.176568 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.176533    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8scm\" (UniqueName: \"kubernetes.io/projected/d7407621-98f1-44a3-8e23-2e7cffc888ec-kube-api-access-z8scm\") pod \"payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:29.268642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.268609    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:29.282808 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.282768    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" event={"ID":"41a9f3ba-4151-4074-b25e-a12adb278946","Type":"ContainerStarted","Data":"bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b"}
Oct 30 20:52:29.282966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.282834    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" event={"ID":"41a9f3ba-4151-4074-b25e-a12adb278946","Type":"ContainerStarted","Data":"45a8e3d35ffc62aa2b9bd566dbf70003cdd5530698e3855bfcd7dd06ffdc6767"}
Oct 30 20:52:29.308816 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.308753    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podStartSLOduration=1.3087373900000001 podStartE2EDuration="1.30873739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:29.30460044 +0000 UTC m=+857.419334629" watchObservedRunningTime="2026-10-30 20:52:29.30873739 +0000 UTC m=+857.423471579"
Oct 30 20:52:29.454699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.454662    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:29.456008 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.455985    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:29.456130 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.456039    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:29.560740 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:29.560701    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7407621_98f1_44a3_8e23_2e7cffc888ec.slice/crio-be107059ff7ba8587621540c31e52e21418ae1be1b4666ec2d1bb5f62b0bb78f WatchSource:0}: Error finding container be107059ff7ba8587621540c31e52e21418ae1be1b4666ec2d1bb5f62b0bb78f: Status 404 returned error can't find the container with id be107059ff7ba8587621540c31e52e21418ae1be1b4666ec2d1bb5f62b0bb78f
Oct 30 20:52:29.564297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.564255    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"]
Oct 30 20:52:29.649059 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.646240    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"]
Oct 30 20:52:29.711946 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.711902    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"]
Oct 30 20:52:29.712127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.712037    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.799890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.799851    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-podinfo\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.799925    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-certs\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.800027    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-envoy\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.800053    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9fb1229-a875-470f-b0a1-342cf3778c47-istiod-ca-cert\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.800081    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-data\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.800112    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.800159    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-token\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.800186    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvvk\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-kube-api-access-nxvvk\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.800317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.800270    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-credential-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901012 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.900926    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-envoy\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901012 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.900963    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9fb1229-a875-470f-b0a1-342cf3778c47-istiod-ca-cert\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901012 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.900982    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-data\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901012 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901009    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901066    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-token\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901096    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvvk\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-kube-api-access-nxvvk\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901188    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-credential-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901256    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-podinfo\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901329    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-certs\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901455    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-data\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901530    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.901772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901753    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9fb1229-a875-470f-b0a1-342cf3778c47-istiod-ca-cert\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.902004 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.901984    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-certs\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.902083 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.902043    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-credential-socket\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.903727 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.903352    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-envoy\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.904108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.904089    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-podinfo\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.924642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.924615    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvvk\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-kube-api-access-nxvvk\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.925184 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.925165    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-token\") pod \"e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") " pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:29.969974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:29.969897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:30.073418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.058005    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:30.157650 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.151734    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:30.246814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.246744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:30.302599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.302189    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn" event={"ID":"d7407621-98f1-44a3-8e23-2e7cffc888ec","Type":"ContainerStarted","Data":"ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da"}
Oct 30 20:52:30.302599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.302237    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:30.302599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.302253    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn" event={"ID":"d7407621-98f1-44a3-8e23-2e7cffc888ec","Type":"ContainerStarted","Data":"be107059ff7ba8587621540c31e52e21418ae1be1b4666ec2d1bb5f62b0bb78f"}
Oct 30 20:52:30.321564 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.321518    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn" podStartSLOduration=2.321503437 podStartE2EDuration="2.321503437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:30.319687193 +0000 UTC m=+858.434421385" watchObservedRunningTime="2026-10-30 20:52:30.321503437 +0000 UTC m=+858.436237626"
Oct 30 20:52:30.468079 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.467939    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:30.468720 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.468683    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:30.517404 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.517358    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"]
Oct 30 20:52:30.517853 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:30.517826    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fb1229_a875_470f_b0a1_342cf3778c47.slice/crio-8887ceafac9ee66ebacff6284b407ee0620e873cf55b4d0f25b79ff7a6f5776c WatchSource:0}: Error finding container 8887ceafac9ee66ebacff6284b407ee0620e873cf55b4d0f25b79ff7a6f5776c: Status 404 returned error can't find the container with id 8887ceafac9ee66ebacff6284b407ee0620e873cf55b4d0f25b79ff7a6f5776c
Oct 30 20:52:30.520452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.520420    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:30.520538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.520488    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:30.520538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:30.520516    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:31.041264 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.041230    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:31.164604 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.161889    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:31.292471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.282913    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:31.423562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.422600    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" event={"ID":"a9fb1229-a875-470f-b0a1-342cf3778c47","Type":"ContainerStarted","Data":"da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22"}
Oct 30 20:52:31.423562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.423072    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" event={"ID":"a9fb1229-a875-470f-b0a1-342cf3778c47","Type":"ContainerStarted","Data":"8887ceafac9ee66ebacff6284b407ee0620e873cf55b4d0f25b79ff7a6f5776c"}
Oct 30 20:52:31.466512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.461496    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:31.466512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.461557    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:31.512411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:31.511535    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podStartSLOduration=2.511514813 podStartE2EDuration="2.511514813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:31.461408558 +0000 UTC m=+859.576142745" watchObservedRunningTime="2026-10-30 20:52:31.511514813 +0000 UTC m=+859.626249004"
Oct 30 20:52:32.105412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.104678    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:32.141445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.106427    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:32.141445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.106519    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:32.187410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.173697    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:32.270419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.269155    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:32.508499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.504897    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:32.508499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.504953    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:32.530410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.524901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:32.804484 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.801518    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"]
Oct 30 20:52:32.830429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.807002    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:32.849424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.844213    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"]
Oct 30 20:52:32.909430 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.867758    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 20:52:32.958428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.945657    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:32.958428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.945739    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxkpf\" (UniqueName: \"kubernetes.io/projected/817e8b83-d053-49d3-901b-d0220b10b9ac-kube-api-access-jxkpf\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:32.958428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.945934    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-xks-ca\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.016497 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:32.981252    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 20:52:33.237095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.188868    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:33.237095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.188943    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:33.333659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.332620    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.333659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.332734    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxkpf\" (UniqueName: \"kubernetes.io/projected/817e8b83-d053-49d3-901b-d0220b10b9ac-kube-api-access-jxkpf\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.333659 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.333118    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-xks-ca\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.408688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.408648    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.412321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.412216    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxkpf\" (UniqueName: \"kubernetes.io/projected/817e8b83-d053-49d3-901b-d0220b10b9ac-kube-api-access-jxkpf\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.412856 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.412800    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-xks-ca\") pod \"maas-api-revoke-sub-63ba1000-rshkj\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.527438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.508582    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:33.527438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.508673    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:33.607756 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.582491    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:33.642478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.610908    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:33.725782 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.725622    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:33.761414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.726842    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:33.815414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.727566    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:33.815414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.727807    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:33.815414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.766503    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:33.853836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.769513    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:33.854351 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:33.782039    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:52:34.105806 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.105768    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:34.106048 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.106017    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:34.320700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.320557    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:52:34.322921 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.322723    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:34.454821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.454780    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:34.455410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.455208    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:34.560433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.560356    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:52:34.564424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.560835    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:52:34.606496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.602889    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:34.606496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.605241    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:52:34.619343 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.619292    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:52:34.620346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.620323    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:52:34.730543 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.730490    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:34.731223 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.731197    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:52:34.803568 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.803523    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:34.804228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.804198    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 20:52:34.804833 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.804812    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:52:34.805046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.805018    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:52:34.805999 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.805978    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:52:35.253438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.814200    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.262352 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.876254    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.304696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:34.876294    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 20:52:35.414410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.408417    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:35.414410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.408533    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:35.434573 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.276906    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:52:35.435164 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.435130    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:52:35.515411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.490007    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:35.636411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.588260    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:35.637328 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.637284    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.637709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.637682    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 20:52:35.637998 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.637974    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:52:35.641409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.638219    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.228680    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.732315    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.739017    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.770010    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.772439    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.772679    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.772938    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:52:35.773965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.773021    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:52:35.793121 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.793050    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.841129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.799963    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:35.849418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.846030    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:52:35.927563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.859471    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 20:52:36.154474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.965498    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:52:36.154474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:35.965599    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:36.188410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.163975    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:36.338689 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.332424    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:36.423479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.404600    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:52:36.423479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.404807    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:52:36.423479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.404933    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:36.477568 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.477530    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:36.477839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.477809    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:36.487791 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.480424    4852 patch_prober.go:29] interesting pod/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 20:52:36.487791 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.480497    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:36.581053 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.497909    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:36.731708 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.710466    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:52:36.764701 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.733038    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:36.864550 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.864512    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:36.896522 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.872507    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:36.938785 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.928602    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:52:36.938785 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.928659    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 20:52:36.938785 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.929573    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:36.938785 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.929610    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:37.075509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:36.974198    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"]
Oct 30 20:52:37.124704 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.104686    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:37.139522 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.139481    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:37.211429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.199987    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 20:52:37.276411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.250265    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:37.276411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.250947    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:52:37.334711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.334607    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:37.465769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.407922    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:37.477565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.477532    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:37.477781 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.477754    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:37.513707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.513678    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:52:37.513941 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.513931    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.525880    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.527208    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj" event={"ID":"817e8b83-d053-49d3-901b-d0220b10b9ac","Type":"ContainerStarted","Data":"c50a721098953895525e94505c879c8365896695736607a006b20e7409858820"}
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.528719    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.529135    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.529698    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.530437    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.531985    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 20:52:37.543419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.538151    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:52:37.615862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.614662    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:52:37.618572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:37.617698    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:52:38.020199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:38.020169    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:38.105603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:38.105536    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:38.105806 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:38.105617    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:38.477419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:38.476645    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 20:52:38.477419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:38.476728    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 20:52:38.539309 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:38.539268    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj" event={"ID":"817e8b83-d053-49d3-901b-d0220b10b9ac","Type":"ContainerStarted","Data":"4bf9a0338bc441a9c16cf5aab0a024e7d6396f3a8ef05359671ea19120431de0"}
Oct 30 20:52:38.584014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:38.583950    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj" podStartSLOduration=6.583932806 podStartE2EDuration="6.583932806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:38.565829073 +0000 UTC m=+866.680563264" watchObservedRunningTime="2026-10-30 20:52:38.583932806 +0000 UTC m=+866.698666997"
Oct 30 20:52:39.104575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.104537    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:39.105020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.104595    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:39.182412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.182231    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:39.262446 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.262411    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:39.371732 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.371629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:39.592551 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.592501    4852 generic.go:356] "Generic (PLEG): container finished" podID="817e8b83-d053-49d3-901b-d0220b10b9ac" containerID="4bf9a0338bc441a9c16cf5aab0a024e7d6396f3a8ef05359671ea19120431de0" exitCode=0
Oct 30 20:52:39.592767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.592583    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj" event={"ID":"817e8b83-d053-49d3-901b-d0220b10b9ac","Type":"ContainerDied","Data":"4bf9a0338bc441a9c16cf5aab0a024e7d6396f3a8ef05359671ea19120431de0"}
Oct 30 20:52:39.770549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.770496    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:39.771074 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.771032    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:39.771904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:39.771881    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:52:40.105225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.105141    4852 patch_prober.go:29] interesting pod/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 20:52:40.105225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.105198    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 20:52:40.208706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.208667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:40.812476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.812367    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:40.912959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.912575    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-openshift-ca\") pod \"817e8b83-d053-49d3-901b-d0220b10b9ac\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") "
Oct 30 20:52:40.912959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.912673    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-xks-ca\") pod \"817e8b83-d053-49d3-901b-d0220b10b9ac\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") "
Oct 30 20:52:40.912959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.912775    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/817e8b83-d053-49d3-901b-d0220b10b9ac-kube-api-access-jxkpf\" (UniqueName: \"kubernetes.io/projected/817e8b83-d053-49d3-901b-d0220b10b9ac-kube-api-access-jxkpf\") pod \"817e8b83-d053-49d3-901b-d0220b10b9ac\" (UID: \"817e8b83-d053-49d3-901b-d0220b10b9ac\") "
Oct 30 20:52:40.913764 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.913696    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-openshift-ca" pod "817e8b83-d053-49d3-901b-d0220b10b9ac" (UID: "817e8b83-d053-49d3-901b-d0220b10b9ac"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:52:40.915669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.915628    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-xks-ca" pod "817e8b83-d053-49d3-901b-d0220b10b9ac" (UID: "817e8b83-d053-49d3-901b-d0220b10b9ac"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:52:40.916177 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:40.916140    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/817e8b83-d053-49d3-901b-d0220b10b9ac-kube-api-access-jxkpf" pod "817e8b83-d053-49d3-901b-d0220b10b9ac" (UID: "817e8b83-d053-49d3-901b-d0220b10b9ac"). InnerVolumeSpecName "kube-api-access-jxkpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:52:41.016447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.014844    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:41.016447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.014887    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxkpf\" (UniqueName: \"kubernetes.io/projected/817e8b83-d053-49d3-901b-d0220b10b9ac-kube-api-access-jxkpf\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:41.016447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.014905    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/817e8b83-d053-49d3-901b-d0220b10b9ac-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:52:41.041875 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.041832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:41.397730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.397669    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:41.398268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.398238    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:41.399015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.398995    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 20:52:41.424873 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.424842    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:52:41.615563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.615536    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"
Oct 30 20:52:41.615737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.615609    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj" event={"ID":"817e8b83-d053-49d3-901b-d0220b10b9ac","Type":"ContainerDied","Data":"c50a721098953895525e94505c879c8365896695736607a006b20e7409858820"}
Oct 30 20:52:41.615737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.615654    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c50a721098953895525e94505c879c8365896695736607a006b20e7409858820"
Oct 30 20:52:41.993736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:41.993681    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:42.867719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:42.867663    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:42.975777 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:42.975743    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:43.688811 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:43.688773    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:44.090929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.090897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:44.485635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.485562    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"]
Oct 30 20:52:44.486138 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.486124    4852 cpu_manager.go:414] "containerMap: removing container" podUID="817e8b83-d053-49d3-901b-d0220b10b9ac" containerName="revoke-keys"
Oct 30 20:52:44.486185 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.486140    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="817e8b83-d053-49d3-901b-d0220b10b9ac" containerName="revoke-keys"
Oct 30 20:52:44.486271 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.486262    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="817e8b83-d053-49d3-901b-d0220b10b9ac" containerName="revoke-keys"
Oct 30 20:52:44.490186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.490159    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.498414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.498271    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-95a684-openshift-default-dockercfg-bs2q6\""
Oct 30 20:52:44.503793 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.503773    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"]
Oct 30 20:52:44.520008 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.519970    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:44.616413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.598461    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-token\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.626408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.617428    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-credential-socket\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.639572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.639527    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-podinfo\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.639884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.639861    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnnr5\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-kube-api-access-dnnr5\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.640056 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.640040    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-socket\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.640189 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.640174    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-data\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.649425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.640501    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-certs\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.649721 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.649696    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-envoy\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.649884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.649842    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499d2e0d-ebae-4f71-9e76-47f307049b15-istiod-ca-cert\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.698901 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.698861    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:44.751465 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751360    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-socket\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.751465 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751431    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-data\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.751718 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751536    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-certs\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.751718 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751560    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-envoy\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.751718 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751583    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499d2e0d-ebae-4f71-9e76-47f307049b15-istiod-ca-cert\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.751876 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751736    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-token\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.751876 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751803    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-socket\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.751876 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751817    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-credential-socket\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.752036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751870    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-data\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.752036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751901    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-certs\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.752036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.751938    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-podinfo\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.752036 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.752018    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnnr5\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-kube-api-access-dnnr5\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.752213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.752054    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-credential-socket\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.752310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.752288    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499d2e0d-ebae-4f71-9e76-47f307049b15-istiod-ca-cert\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.754225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.754200    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-envoy\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.754381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.754363    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-podinfo\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.767134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.767098    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-token\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.770230 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.770204    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnnr5\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-kube-api-access-dnnr5\") pod \"e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") " pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.813966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.813933    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:44.998947 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:44.998905    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:45.127307 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.124231    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"]
Oct 30 20:52:45.130755 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:45.130722    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499d2e0d_ebae_4f71_9e76_47f307049b15.slice/crio-e8ce8247dac94bc6df180b89b69b5eadb2074b42f89570a4f9e71aeae9271586 WatchSource:0}: Error finding container e8ce8247dac94bc6df180b89b69b5eadb2074b42f89570a4f9e71aeae9271586: Status 404 returned error can't find the container with id e8ce8247dac94bc6df180b89b69b5eadb2074b42f89570a4f9e71aeae9271586
Oct 30 20:52:45.136616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.134151    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:45.136616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.134248    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:45.136616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.134295    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:45.659950 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.659912    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" event={"ID":"499d2e0d-ebae-4f71-9e76-47f307049b15","Type":"ContainerStarted","Data":"81c0c9622ddc3c1c58cf0aca05ffce1e3ba5c4c229d4e9337e1927ba0aedfe6b"}
Oct 30 20:52:45.660149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.659970    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" event={"ID":"499d2e0d-ebae-4f71-9e76-47f307049b15","Type":"ContainerStarted","Data":"e8ce8247dac94bc6df180b89b69b5eadb2074b42f89570a4f9e71aeae9271586"}
Oct 30 20:52:45.682630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.682578    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podStartSLOduration=1.68256316 podStartE2EDuration="1.68256316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:45.678438332 +0000 UTC m=+873.793172517" watchObservedRunningTime="2026-10-30 20:52:45.68256316 +0000 UTC m=+873.797297350"
Oct 30 20:52:45.814583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.814545    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:45.845579 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:45.845541    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:46.818699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:46.818645    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:46.820769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:46.820746    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:47.769426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:47.768877    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:47.796940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:47.796538    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:52:48.043601 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:48.043510    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:48.152552 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:48.152511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:49.038959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.038922    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:49.394997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.394885    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:49.480115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.480080    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"]
Oct 30 20:52:49.484340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.484314    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:49.494018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.493989    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"]
Oct 30 20:52:49.512382 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.512340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:49.537951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.537904    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/604a5929-5208-4d37-84c8-49a113f22dd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:49.538179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.538035    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kksvj\" (UniqueName: \"kubernetes.io/projected/604a5929-5208-4d37-84c8-49a113f22dd4-kube-api-access-kksvj\") pod \"payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:49.641706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.638171    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-95a684\""
Oct 30 20:52:49.641706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.638311    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-95a684-dockercfg-749fk\""
Oct 30 20:52:49.659469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.650848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:49.659469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.651296    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kksvj\" (UniqueName: \"kubernetes.io/projected/604a5929-5208-4d37-84c8-49a113f22dd4-kube-api-access-kksvj\") pod \"payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:49.659469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.651805    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/604a5929-5208-4d37-84c8-49a113f22dd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:49.659469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.657707    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/604a5929-5208-4d37-84c8-49a113f22dd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:49.661510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.661482    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kksvj\" (UniqueName: \"kubernetes.io/projected/604a5929-5208-4d37-84c8-49a113f22dd4-kube-api-access-kksvj\") pod \"payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:49.815922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:49.815889    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:50.048140 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.048110    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:50.112945 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.104126    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"]
Oct 30 20:52:50.400433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.400311    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:50.500549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.500520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:50.805020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.804980    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd" event={"ID":"604a5929-5208-4d37-84c8-49a113f22dd4","Type":"ContainerStarted","Data":"da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca"}
Oct 30 20:52:50.805221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.805050    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd" event={"ID":"604a5929-5208-4d37-84c8-49a113f22dd4","Type":"ContainerStarted","Data":"1274dbf227958d21c1217eb0803310a7c541c20f62d9afef6304dd9dc4e89219"}
Oct 30 20:52:50.805221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.805194    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:52:50.827983 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.827932    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd" podStartSLOduration=1.827919394 podStartE2EDuration="1.827919394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:50.823708852 +0000 UTC m=+878.938443044" watchObservedRunningTime="2026-10-30 20:52:50.827919394 +0000 UTC m=+878.942653584"
Oct 30 20:52:50.837559 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:50.837522    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:51.559068 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:52:51.559033    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:52:51.641511 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:51.641473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:52.248299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:52.248266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:52.577310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:52.577274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:52.717696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:52.717664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:52.873690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:52.873299    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:53.233734 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.233642    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:53.624914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.624882    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"]
Oct 30 20:52:53.629675 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.629657    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.638075 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.637651    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-49990a-openshift-default-dockercfg-kbqrs\""
Oct 30 20:52:53.644613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.644587    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"]
Oct 30 20:52:53.677178 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.677144    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:53.708747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.708709    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-token\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.708945 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.708764    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-certs\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.709017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.708994    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-podinfo\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.709122 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.709093    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-socket\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.709246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.709147    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-envoy\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.709246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.709181    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istiod-ca-cert\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.709246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.709234    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-credential-socket\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.709455 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.709252    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-data\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.709455 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.709318    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spfb5\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-kube-api-access-spfb5\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.813579    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spfb5\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-kube-api-access-spfb5\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.813674    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-token\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.813705    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-certs\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.813803    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-podinfo\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.813874    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-socket\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.813940    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-envoy\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.813992    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istiod-ca-cert\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.814074    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-credential-socket\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.814424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.814103    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-data\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.825470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.815212    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-certs\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.825470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.821756    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istiod-ca-cert\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.825470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.822042    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-socket\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.825470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.824023    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-credential-socket\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.825792 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.825541    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-data\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.829653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.829599    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-podinfo\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.830307 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.830287    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-envoy\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.847436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.847379    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-token\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.848232 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.848209    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spfb5\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-kube-api-access-spfb5\") pod \"e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") " pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:53.945850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:53.945768    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:54.068327 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.068291    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:54.168740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.168702    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:54.243276 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:52:54.243237    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591a15d1_0222_4ea1_b512_7c4ac5d726c2.slice/crio-4676d7ca223baf06d31a5c031fd0e502341d3309ededb60e4fc92db52ee08751 WatchSource:0}: Error finding container 4676d7ca223baf06d31a5c031fd0e502341d3309ededb60e4fc92db52ee08751: Status 404 returned error can't find the container with id 4676d7ca223baf06d31a5c031fd0e502341d3309ededb60e4fc92db52ee08751
Oct 30 20:52:54.244354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.244298    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"]
Oct 30 20:52:54.246306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.246272    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:54.246429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.246403    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:54.246508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.246453    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:52:54.915434 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.915322    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" event={"ID":"591a15d1-0222-4ea1-b512-7c4ac5d726c2","Type":"ContainerStarted","Data":"287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c"}
Oct 30 20:52:54.916009 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.915485    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" event={"ID":"591a15d1-0222-4ea1-b512-7c4ac5d726c2","Type":"ContainerStarted","Data":"4676d7ca223baf06d31a5c031fd0e502341d3309ededb60e4fc92db52ee08751"}
Oct 30 20:52:54.948885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.948474    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:54.973658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:54.973596    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" podStartSLOduration=1.973574817 podStartE2EDuration="1.973574817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:52:54.949826714 +0000 UTC m=+883.064560923" watchObservedRunningTime="2026-10-30 20:52:54.973574817 +0000 UTC m=+883.088309009"
Oct 30 20:52:55.004261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:55.004227    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:55.147608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:55.147577    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:55.982410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:55.970965    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:55.982410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:55.972846    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:55.982410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:55.981477    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:52:57.157979 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.157943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:57.323495 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.299899    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:57.362458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.361523    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 20:52:57.492513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.482961    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:57.492513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.491482    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:57.537658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.537609    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 20:52:57.538194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.538166    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:52:57.554965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.554869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:57.667537 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.667486    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:57.769283 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.767688    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:57.917304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:57.917263    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:58.118162 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:58.117964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:58.316708 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:58.311951    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:52:58.946412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:58.945516    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:58.946412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:58.945929    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:52:58.946412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:58.946205    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:52:59.234492 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.181817    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:59.461419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.399288    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:59.526526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.493809    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:59.526526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.509778    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:59.570697 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.570637    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:59.632412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.583980    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:52:59.981015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.927164    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 20:52:59.981015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.928160    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 20:52:59.981015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:52:59.928575    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:53:00.097331 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.097279    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 20:53:00.274471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.236735    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:00.274471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.238774    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:53:00.274471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.241865    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 20:53:00.302897 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.302843    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:53:00.337414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.337363    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:00.402631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.402568    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/health\": context deadline exceeded"
Oct 30 20:53:00.429317 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.429286    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:53:00.429603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.429589    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:53:00.429715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.429706    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:53:00.476379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.475016    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:53:00.478156 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.477188    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:53:00.507062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.507011    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:53:00.513623 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.513591    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:53:00.557541 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.557453    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"]
Oct 30 20:53:00.801011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.759190    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:53:00.801011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.778925    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:53:00.801011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.779692    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:00.801011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.786476    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a52bd2a1\"/\"e2e-lifecycle-model-a52bd2a1-kserve-self-signed-certs\""
Oct 30 20:53:00.801011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.786610    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a52bd2a1\"/\"default-dockercfg-d6fdv\""
Oct 30 20:53:00.801011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.786680    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a52bd2a1\"/\"openshift-service-ca.crt\""
Oct 30 20:53:00.801011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.786766    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a52bd2a1\"/\"kube-root-ca.crt\""
Oct 30 20:53:00.810038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.809959    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 20:53:00.885410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.881983    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tls-certs\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:00.885410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.882076    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-dshm\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:00.885410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.882137    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tmp-dir\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:00.885410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.882297    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-home\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:00.885410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.882355    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch6jx\" (UniqueName: \"kubernetes.io/projected/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-kube-api-access-ch6jx\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:00.885410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.882411    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-model-cache\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:00.925419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.919861    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"]
Oct 30 20:53:01.018416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.935187    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:53:01.047900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:00.956644    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"]
Oct 30 20:53:01.127473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.110115    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:53:01.252575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.006517    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tls-certs\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.259542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.259479    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-dshm\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.263410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.259850    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tmp-dir\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.346280 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.346240    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-home\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.346488 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.346327    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch6jx\" (UniqueName: \"kubernetes.io/projected/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-kube-api-access-ch6jx\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.346488 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.346370    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-model-cache\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.348890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.178849    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 20:53:01.348890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.347458    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"]
Oct 30 20:53:01.348890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.347587    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 20:53:01.348890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.347624    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 20:53:01.348890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.347653    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 20:53:01.348890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.347690    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:53:01.367500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.367470    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 20:53:01.396488 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.199197    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tls-certs\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.397040 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.201774    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:53:01.401010 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.400973    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-model-cache\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.401195 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.135289    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:01.401381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.329043    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tmp-dir\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.404797 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.404772    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-home\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.411989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.411961    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 20:53:01.412197 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.132047    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.436365 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.421234    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1982ff\""
Oct 30 20:53:01.439533 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.439497    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1982ff-dockercfg-zf4k4\""
Oct 30 20:53:01.439838 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.439815    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 20:53:01.440067 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.440049    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-dshm\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.452508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.451286    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 20:53:01.503795 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.501814    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba7bd9a-e82b-4940-8dd5-9e244029a134-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.503795 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.501890    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gcd\" (UniqueName: \"kubernetes.io/projected/bba7bd9a-e82b-4940-8dd5-9e244029a134-kube-api-access-p2gcd\") pod \"payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.506432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.506333    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch6jx\" (UniqueName: \"kubernetes.io/projected/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-kube-api-access-ch6jx\") pod \"e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") " pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.581750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.581718    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:01.604046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.604011    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba7bd9a-e82b-4940-8dd5-9e244029a134-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.604211 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.604112    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gcd\" (UniqueName: \"kubernetes.io/projected/bba7bd9a-e82b-4940-8dd5-9e244029a134-kube-api-access-p2gcd\") pod \"payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.607430 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.607239    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba7bd9a-e82b-4940-8dd5-9e244029a134-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.623892 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.623856    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gcd\" (UniqueName: \"kubernetes.io/projected/bba7bd9a-e82b-4940-8dd5-9e244029a134-kube-api-access-p2gcd\") pod \"payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.805936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.805712    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:01.822182 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.822149    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:53:01.923981 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:01.923853    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"]
Oct 30 20:53:02.133595 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.132964    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"]
Oct 30 20:53:02.147747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.147683    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:02.235649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.235612    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:02.283376 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.283339    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"]
Oct 30 20:53:02.287396 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.287365    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:02.304009 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.303971    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"]
Oct 30 20:53:02.314896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.314867    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db620f75-26b3-4049-aaa7-20e403e1c753-plugins-config-volume\") pod \"payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") " pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:02.315050 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.315030    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g89l\" (UniqueName: \"kubernetes.io/projected/db620f75-26b3-4049-aaa7-20e403e1c753-kube-api-access-8g89l\") pod \"payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") " pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:02.343337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.343311    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-49990a\""
Oct 30 20:53:02.343573 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.343437    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-49990a-dockercfg-nnvsm\""
Oct 30 20:53:02.401597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.401564    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:02.416278 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.416241    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g89l\" (UniqueName: \"kubernetes.io/projected/db620f75-26b3-4049-aaa7-20e403e1c753-kube-api-access-8g89l\") pod \"payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") " pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:02.416487 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.416371    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db620f75-26b3-4049-aaa7-20e403e1c753-plugins-config-volume\") pod \"payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") " pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:02.417072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.417047    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db620f75-26b3-4049-aaa7-20e403e1c753-plugins-config-volume\") pod \"payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") " pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:02.425687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.425655    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g89l\" (UniqueName: \"kubernetes.io/projected/db620f75-26b3-4049-aaa7-20e403e1c753-kube-api-access-8g89l\") pod \"payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") " pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:02.443684 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.443634    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756" event={"ID":"bba7bd9a-e82b-4940-8dd5-9e244029a134","Type":"ContainerStarted","Data":"01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca"}
Oct 30 20:53:02.443846 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.443698    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756" event={"ID":"bba7bd9a-e82b-4940-8dd5-9e244029a134","Type":"ContainerStarted","Data":"3e348a616c76e7535dc96817d88a9e7ea03da79c021379a25561f75558355454"}
Oct 30 20:53:02.443945 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.443917    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:02.446028 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.446004    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" event={"ID":"6f74e3ec-1cc6-4223-9783-bef1f4fea17f","Type":"ContainerStarted","Data":"2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04"}
Oct 30 20:53:02.446163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.446040    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" event={"ID":"6f74e3ec-1cc6-4223-9783-bef1f4fea17f","Type":"ContainerStarted","Data":"b6b0bd4594409c6e213ce2129023d57030285870e4e0e2ab615ac5fe538526e4"}
Oct 30 20:53:02.500478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.499767    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756" podStartSLOduration=3.4997452620000002 podStartE2EDuration="3.499745262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:53:02.496797906 +0000 UTC m=+890.611532095" watchObservedRunningTime="2026-10-30 20:53:02.499745262 +0000 UTC m=+890.614479455"
Oct 30 20:53:02.567447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.567183    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" podStartSLOduration=3.567162538 podStartE2EDuration="3.567162538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:53:02.546998909 +0000 UTC m=+890.661733100" watchObservedRunningTime="2026-10-30 20:53:02.567162538 +0000 UTC m=+890.681896728"
Oct 30 20:53:02.591428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.587241    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:02.610432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:02.609737    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:03.031428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:03.031361    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"]
Oct 30 20:53:03.031668 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:53:03.031643    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb620f75_26b3_4049_aaa7_20e403e1c753.slice/crio-64cc3759d04bf8f7df169adfdee6426dc9a10b54fde4b4f5ae420773bfe48c66 WatchSource:0}: Error finding container 64cc3759d04bf8f7df169adfdee6426dc9a10b54fde4b4f5ae420773bfe48c66: Status 404 returned error can't find the container with id 64cc3759d04bf8f7df169adfdee6426dc9a10b54fde4b4f5ae420773bfe48c66
Oct 30 20:53:03.399804 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:03.399677    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:03.512412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:03.506350    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92" event={"ID":"db620f75-26b3-4049-aaa7-20e403e1c753","Type":"ContainerStarted","Data":"0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0"}
Oct 30 20:53:03.512412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:03.506491    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92" event={"ID":"db620f75-26b3-4049-aaa7-20e403e1c753","Type":"ContainerStarted","Data":"64cc3759d04bf8f7df169adfdee6426dc9a10b54fde4b4f5ae420773bfe48c66"}
Oct 30 20:53:03.512412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:03.507242    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:03.512412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:03.507619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:03.535022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:03.534971    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92" podStartSLOduration=1.534957038 podStartE2EDuration="1.534957038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:53:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:53:03.530140056 +0000 UTC m=+891.644874257" watchObservedRunningTime="2026-10-30 20:53:03.534957038 +0000 UTC m=+891.649691236"
Oct 30 20:53:04.542898 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:04.542845    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:05.424834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:05.424755    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:06.264599 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:06.264529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:07.110743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:07.110710    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:07.908696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:07.908601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:08.217593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.217490    4852 patch_prober.go:29] interesting pod/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 20:53:08.217593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.217564    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 20:53:08.554791 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.554762    4852 generic.go:356] "Generic (PLEG): container finished" podID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerID="30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4" exitCode=0
Oct 30 20:53:08.554968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.554836    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" event={"ID":"061e7d86-7f1c-40db-8ecd-76ce35bcadee","Type":"ContainerDied","Data":"30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4"}
Oct 30 20:53:08.555310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.555295    4852 scope.go:122] "RemoveContainer" containerID="30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4"
Oct 30 20:53:08.557221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.557186    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:08.557353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.557277    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:08.557353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.557325    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:08.607820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.607769    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"]
Oct 30 20:53:08.618079 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.618047    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.626472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.626446    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1982ff-serving-cert\""
Oct 30 20:53:08.645060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.645036    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"]
Oct 30 20:53:08.650815 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.650730    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:08.656249 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.654695    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.659529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.659503    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"]
Oct 30 20:53:08.664307 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.664287    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"]
Oct 30 20:53:08.745413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.738704    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1fac16-openshift-default-dockercfg-vvhxw\""
Oct 30 20:53:08.764882 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.764848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:08.788341 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788305    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhbmm\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-kube-api-access-nhbmm\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788360    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-token\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788460    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-podinfo\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788508    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-envoy\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788547    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-data\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788813 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788695    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-certs\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788813 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788765    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-maas-api-tls\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.788919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788811    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-socket\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788839    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istiod-ca-cert\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.788919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788908    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-metrics-certs\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.789064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788938    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3d46f4-7d9d-4166-95de-23f794b86212-openshift-service-ca\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.789064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.788999    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9mbv\" (UniqueName: \"kubernetes.io/projected/ff3d46f4-7d9d-4166-95de-23f794b86212-kube-api-access-j9mbv\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.789064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.789029    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-credential-socket\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.861586 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.861496    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:08.890770 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.890729    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhbmm\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-kube-api-access-nhbmm\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.890957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.890783    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-token\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.890957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.890821    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-podinfo\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.890957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.890865    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-envoy\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.890957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.890906    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-data\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.891172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891006    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-certs\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.891172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891068    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-maas-api-tls\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.891172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891128    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-socket\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.891172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891165    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istiod-ca-cert\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.891371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891230    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-metrics-certs\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.891371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891258    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3d46f4-7d9d-4166-95de-23f794b86212-openshift-service-ca\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.891530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891371    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9mbv\" (UniqueName: \"kubernetes.io/projected/ff3d46f4-7d9d-4166-95de-23f794b86212-kube-api-access-j9mbv\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.891530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891431    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-credential-socket\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.891899 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.891868    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-data\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.892446 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.892415    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3d46f4-7d9d-4166-95de-23f794b86212-openshift-service-ca\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.892827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.892603    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-certs\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.892827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.892784    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-socket\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.893319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.893298    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-credential-socket\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.893511 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.893348    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istiod-ca-cert\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.893926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.893899    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-envoy\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.894124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.894104    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-podinfo\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.894881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.894862    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-maas-api-tls\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.895089 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.895070    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-metrics-certs\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.900366 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.900336    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-token\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.900531 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.900059    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhbmm\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-kube-api-access-nhbmm\") pod \"e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") " pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:08.901339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.901316    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9mbv\" (UniqueName: \"kubernetes.io/projected/ff3d46f4-7d9d-4166-95de-23f794b86212-kube-api-access-j9mbv\") pod \"maas-api-e2e-api-a-1982ff-d679d459f-mkbr8\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.949270 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.949240    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:08.983038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:08.983001    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:09.284186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.284147    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"]
Oct 30 20:53:09.293251 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:53:09.293214    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3d46f4_7d9d_4166_95de_23f794b86212.slice/crio-93e95606fe7201b45522de32def15315a060b772582acc2a0d500d442d71add6 WatchSource:0}: Error finding container 93e95606fe7201b45522de32def15315a060b772582acc2a0d500d442d71add6: Status 404 returned error can't find the container with id 93e95606fe7201b45522de32def15315a060b772582acc2a0d500d442d71add6
Oct 30 20:53:09.297360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.296311    4852 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:53:09.329971 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:53:09.329923    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59a1f96_18d7_48a9_85a9_fc5fd0027950.slice/crio-9b37e567dfa4ea80bd7e276570b5cf08ad7adea05793d5423b1e209585057816 WatchSource:0}: Error finding container 9b37e567dfa4ea80bd7e276570b5cf08ad7adea05793d5423b1e209585057816: Status 404 returned error can't find the container with id 9b37e567dfa4ea80bd7e276570b5cf08ad7adea05793d5423b1e209585057816
Oct 30 20:53:09.331845 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.331823    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"]
Oct 30 20:53:09.334524 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.334488    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:09.334630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.334572    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:09.334630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.334605    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:09.596470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.596340    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" event={"ID":"e59a1f96-18d7-48a9-85a9-fc5fd0027950","Type":"ContainerStarted","Data":"44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542"}
Oct 30 20:53:09.596743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.596695    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" event={"ID":"e59a1f96-18d7-48a9-85a9-fc5fd0027950","Type":"ContainerStarted","Data":"9b37e567dfa4ea80bd7e276570b5cf08ad7adea05793d5423b1e209585057816"}
Oct 30 20:53:09.599246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.599219    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8" event={"ID":"ff3d46f4-7d9d-4166-95de-23f794b86212","Type":"ContainerStarted","Data":"93e95606fe7201b45522de32def15315a060b772582acc2a0d500d442d71add6"}
Oct 30 20:53:09.604318 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.604138    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" event={"ID":"061e7d86-7f1c-40db-8ecd-76ce35bcadee","Type":"ContainerStarted","Data":"81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f"}
Oct 30 20:53:09.624507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.624322    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" podStartSLOduration=1.624301534 podStartE2EDuration="1.624301534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:53:09.621900921 +0000 UTC m=+897.736635113" watchObservedRunningTime="2026-10-30 20:53:09.624301534 +0000 UTC m=+897.739035727"
Oct 30 20:53:09.664411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.661941    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:09.785713 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.774239    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:09.878751 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.874913    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:09.983719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:09.983682    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:10.576410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.575765    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:53:10.576410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.575895    4852 patch_prober.go:29] interesting pod/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 20:53:10.576410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.575942    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 20:53:10.621462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.621427    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8" event={"ID":"ff3d46f4-7d9d-4166-95de-23f794b86212","Type":"ContainerStarted","Data":"cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14"}
Oct 30 20:53:10.621649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.621510    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:10.643978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.643928    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8" podStartSLOduration=2.212280902 podStartE2EDuration="2.643914311s" totalImagesPullingTime="431.633409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:53:10.641206772 +0000 UTC m=+898.755940962" watchObservedRunningTime="2026-10-30 20:53:10.643914311 +0000 UTC m=+898.758648501"
Oct 30 20:53:10.925256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.925143    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:10.989557 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.989514    4852 patch_prober.go:29] interesting pod/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 20:53:10.990143 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:10.990108    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 20:53:11.582422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:11.575889    4852 patch_prober.go:29] interesting pod/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 20:53:11.582422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:11.575991    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 20:53:11.591416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:11.587247    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:11.637353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:11.637324    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:11.705446 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:11.705292    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:53:12.039283 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:12.039235    4852 patch_prober.go:29] interesting pod/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 20:53:12.039745 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:12.039296    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 20:53:12.212134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:12.212094    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:12.585419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:12.581843    4852 patch_prober.go:29] interesting pod/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 20:53:12.585419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:12.581909    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 20:53:12.991547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:12.987504    4852 patch_prober.go:29] interesting pod/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 20:53:12.991547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:12.987565    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 20:53:13.201242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.201199    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:13.343442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.341945    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"]
Oct 30 20:53:13.347412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.347120    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"]
Oct 30 20:53:13.347412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.347256    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.409449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.403193    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:13.540674 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.540644    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.570554    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.570658    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.570749    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.570837    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.570923    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.570951    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.571045    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npdpd\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-kube-api-access-npdpd\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.571119    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.573411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.571150    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.597470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.583812    4852 patch_prober.go:29] interesting pod/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 20:53:13.597470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.583871    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.706362    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npdpd\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-kube-api-access-npdpd\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.706487    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.706537    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.706647    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.706742    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.706824    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.706952    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.707057    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.707084    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.711534    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.712138    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.712699    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.713320    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.714687    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.715052    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.715054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.715062    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.726678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.726443    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.726835 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.726728    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npdpd\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-kube-api-access-npdpd\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:13.812259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.812225    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:13.990468 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.989812    4852 patch_prober.go:29] interesting pod/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 20:53:13.990468 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:13.989894    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 20:53:14.021006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.020977    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:14.307070 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.306994    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:14.438409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.427491    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"]
Oct 30 20:53:14.438409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.428223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:14.531408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.525151    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:14.542416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.537479    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:14.542416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.537580    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:14.542416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.537622    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:14.581419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.580779    4852 patch_prober.go:29] interesting pod/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 20:53:14.581419 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.580862    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 20:53:14.780412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.773101    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" event={"ID":"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da","Type":"ContainerStarted","Data":"870b37a5e6cc2ba837c4e109106835c1bc660de1997030b561bf8177e0689cf8"}
Oct 30 20:53:14.986411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.984048    4852 patch_prober.go:29] interesting pod/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 20:53:14.986411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:14.984140    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 20:53:15.525016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:15.524979    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:15.529050 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:15.529023    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:53:15.585578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:15.585540    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:53:15.586283 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:15.586259    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:53:15.587403 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:15.587371    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:53:15.802557 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:15.798846    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" event={"ID":"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da","Type":"ContainerStarted","Data":"95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4"}
Oct 30 20:53:15.841410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:15.840085    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podStartSLOduration=2.8400642019999998 podStartE2EDuration="2.840064202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:53:15.832811767 +0000 UTC m=+903.947545958" watchObservedRunningTime="2026-10-30 20:53:15.840064202 +0000 UTC m=+903.954798393"
Oct 30 20:53:16.021569 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.021532    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:16.023271 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.023241    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 20:53:16.023421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.023296    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 20:53:16.160467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.156832    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:16.160467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.157809    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:16.163418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.163193    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:53:16.570434 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.567256    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:16.640715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:16.640678    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:53:17.021565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.021526    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 20:53:17.021800 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.021596    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 20:53:17.514722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.514690    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"]
Oct 30 20:53:17.522824 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.522788    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.530656 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.530627    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"]
Oct 30 20:53:17.531555 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531526    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531579    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531739    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531786    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531836    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531864    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531957    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.531998    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.532091    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.532135    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nwdz\" (UniqueName: \"kubernetes.io/projected/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-kube-api-access-8nwdz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.538477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.532145    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gjgsg\""
Oct 30 20:53:17.633940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.633895    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.633940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.633946    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.634469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.633978    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.634469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.634109    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.634469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.634143    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nwdz\" (UniqueName: \"kubernetes.io/projected/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-kube-api-access-8nwdz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.634469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.634216    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.634469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.634252    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.635075 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.635048    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.636759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.636732    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.636913 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.636886    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.637027 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.636753    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.637300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.637282    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.642841 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.642805    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nwdz\" (UniqueName: \"kubernetes.io/projected/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-kube-api-access-8nwdz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.643149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.643126    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-flf8w\" (UID: \"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:17.879643 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:17.879554    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:18.005378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:18.005048    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:18.021860 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:18.021825    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 20:53:18.022099 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:18.022072    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 20:53:18.293704 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:18.278895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:18.293704 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:18.281780    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"]
Oct 30 20:53:18.826593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:18.826556    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w" event={"ID":"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9","Type":"ContainerStarted","Data":"53c6419de9178f2a023d10036ecd466553ceb1541afd2a365841f675a811c69d"}
Oct 30 20:53:19.021898 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:19.021864    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 20:53:19.022064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:19.021928    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 20:53:19.324264 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:19.324229    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:19.408613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:19.408578    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:19.782710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:19.782349    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:20.021623 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:20.021591    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 20:53:20.022159 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:20.021648    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 20:53:21.022264 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:21.022228    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 20:53:21.022729 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:21.022294    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 20:53:21.377136 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:21.376999    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:21.570191 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:53:21.567651    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:53:22.029209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:22.028784    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 20:53:22.029209 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:22.028845    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 20:53:22.773046 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:22.772963    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:23.208991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:23.208941    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:23.209408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:23.209032    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:23.320252 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:23.320225    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:23.320811 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:23.320791    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:23.322439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:23.322422    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 20:53:24.026438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:24.025969    4852 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-flf8w container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:53:24.026438 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:24.026035    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w" podUID="e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:53:24.036927 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:24.036894    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:24.037100 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:24.036947    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w" event={"ID":"e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9","Type":"ContainerStarted","Data":"143169549b0282dc5fcad0ca3d8dffd0d9fd4e47474dcce090d8c892502ebeca"}
Oct 30 20:53:24.070207 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:24.070138    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w" podStartSLOduration=2.267218732 podStartE2EDuration="7.070115294s" totalImagesPullingTime="4.802896562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:53:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:53:24.047222869 +0000 UTC m=+912.161957083" watchObservedRunningTime="2026-10-30 20:53:24.070115294 +0000 UTC m=+912.184849486"
Oct 30 20:53:24.232014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:24.231971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:25.035032 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:25.035002    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-flf8w"
Oct 30 20:53:26.030104 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:26.030064    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:26.945440 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:26.945405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:27.332331 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:27.332298    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:27.577240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:27.577207    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:27.812504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:27.812455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:28.117295 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:28.117221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:28.274276 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:28.274238    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:28.736652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:28.736619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:29.627794 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:29.627756    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:30.822341 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:30.822302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:31.446060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:31.445971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:31.545873 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:31.545839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:31.939505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:31.939467    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:32.111304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.111274    4852 generic.go:356] "Generic (PLEG): container finished" podID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerID="287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c" exitCode=0
Oct 30 20:53:32.111472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.111346    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" event={"ID":"591a15d1-0222-4ea1-b512-7c4ac5d726c2","Type":"ContainerDied","Data":"287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c"}
Oct 30 20:53:32.111853 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.111838    4852 scope.go:122] "RemoveContainer" containerID="287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c"
Oct 30 20:53:32.113485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.113457    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:32.113597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.113539    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:32.113597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.113590    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:53:32.673146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.673091    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:32.769409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:32.768842    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:33.116409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:33.108715    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:33.131163 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:33.131127    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" event={"ID":"591a15d1-0222-4ea1-b512-7c4ac5d726c2","Type":"ContainerStarted","Data":"df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845"}
Oct 30 20:53:33.201608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:33.201569    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:33.946731 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:33.946697    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:53:33.946964 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:33.946943    4852 patch_prober.go:29] interesting pod/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 20:53:33.947024 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:33.947000    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 20:53:34.525628 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:34.525590    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:34.946871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:34.946786    4852 patch_prober.go:29] interesting pod/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 20:53:34.946871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:34.946850    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 20:53:35.342915 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:35.342879    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:35.946467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:35.946433    4852 patch_prober.go:29] interesting pod/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 20:53:35.946873 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:35.946500    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 20:53:36.365473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:36.355882    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:36.946948 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:36.946914    4852 patch_prober.go:29] interesting pod/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 20:53:36.947346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:36.946976    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 20:53:37.176025 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:37.175986    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:37.946300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:37.946262    4852 patch_prober.go:29] interesting pod/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 20:53:37.946510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:37.946315    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 20:53:37.975250 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:37.975207    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:38.823153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:38.823113    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:38.951840 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:38.951811    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:53:38.952132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:38.952111    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:53:38.952998 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:38.952962    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:53:39.706600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:39.706550    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:40.320575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:40.320540    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"]
Oct 30 20:53:40.370651 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:40.370616    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:40.469827 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:40.469792    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:40.611452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:40.611360    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:41.665508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:41.665473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:42.872809 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:42.872728    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:42.975551 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:42.975498    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:44.151930 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:44.151900    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:45.148187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.148143    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:45.321776 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.321732    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn" podUID="d7407621-98f1-44a3-8e23-2e7cffc888ec" containerName="payload-pre-processing" containerID="cri-o://ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da" gracePeriod=25
Oct 30 20:53:45.609311 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.609286    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:53:45.756572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.756535    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7407621-98f1-44a3-8e23-2e7cffc888ec-kube-api-access-z8scm\" (UniqueName: \"kubernetes.io/projected/d7407621-98f1-44a3-8e23-2e7cffc888ec-kube-api-access-z8scm\") pod \"d7407621-98f1-44a3-8e23-2e7cffc888ec\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") "
Oct 30 20:53:45.756779 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.756757    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7407621-98f1-44a3-8e23-2e7cffc888ec-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7407621-98f1-44a3-8e23-2e7cffc888ec-plugins-config-volume\") pod \"d7407621-98f1-44a3-8e23-2e7cffc888ec\" (UID: \"d7407621-98f1-44a3-8e23-2e7cffc888ec\") "
Oct 30 20:53:45.757133 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.757111    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7407621-98f1-44a3-8e23-2e7cffc888ec-plugins-config-volume" pod "d7407621-98f1-44a3-8e23-2e7cffc888ec" (UID: "d7407621-98f1-44a3-8e23-2e7cffc888ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:53:45.757492 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.757474    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7407621-98f1-44a3-8e23-2e7cffc888ec-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:53:45.758804 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.758783    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7407621-98f1-44a3-8e23-2e7cffc888ec-kube-api-access-z8scm" pod "d7407621-98f1-44a3-8e23-2e7cffc888ec" (UID: "d7407621-98f1-44a3-8e23-2e7cffc888ec"). InnerVolumeSpecName "kube-api-access-z8scm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:53:45.858621 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:45.858589    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8scm\" (UniqueName: \"kubernetes.io/projected/d7407621-98f1-44a3-8e23-2e7cffc888ec-kube-api-access-z8scm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:53:46.130637 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.130594    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:46.309794 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.309710    4852 generic.go:356] "Generic (PLEG): container finished" podID="d7407621-98f1-44a3-8e23-2e7cffc888ec" containerID="ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da" exitCode=0
Oct 30 20:53:46.309794 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.309755    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"
Oct 30 20:53:46.309995 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.309788    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn" event={"ID":"d7407621-98f1-44a3-8e23-2e7cffc888ec","Type":"ContainerDied","Data":"ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da"}
Oct 30 20:53:46.309995 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.309848    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn" event={"ID":"d7407621-98f1-44a3-8e23-2e7cffc888ec","Type":"ContainerDied","Data":"be107059ff7ba8587621540c31e52e21418ae1be1b4666ec2d1bb5f62b0bb78f"}
Oct 30 20:53:46.309995 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.309874    4852 scope.go:122] "RemoveContainer" containerID="ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da"
Oct 30 20:53:46.325199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.325175    4852 scope.go:122] "RemoveContainer" containerID="ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da"
Oct 30 20:53:46.326058 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:53:46.325677    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da\": container with ID starting with ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da not found: ID does not exist" containerID="ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da"
Oct 30 20:53:46.326058 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.325740    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da"} err="failed to get container status \"ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da\": rpc error: code = NotFound desc = could not find container \"ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da\": container with ID starting with ca233e12d6be725acd9b55e7d053178e617aa1a6d9b4673da7244837043673da not found: ID does not exist"
Oct 30 20:53:46.332803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.332736    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"]
Oct 30 20:53:46.338230 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:46.338208    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1fac16-645f4f8ff5-hl7cn"]
Oct 30 20:53:47.107144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:47.107111    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:47.208099 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:47.207826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:47.310826 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:47.310792    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:48.005896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:48.005861    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7407621-98f1-44a3-8e23-2e7cffc888ec" path="/var/lib/kubelet/pods/d7407621-98f1-44a3-8e23-2e7cffc888ec/volumes"
Oct 30 20:53:48.252741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:48.252711    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:48.360221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:48.360124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:49.144335 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:49.144295    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:50.298785 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:50.298738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:51.171269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:51.171233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:51.560189 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:53:51.560156    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:53:51.992608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:51.992524    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:52.823571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:52.823538    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:53.768891 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:53.768852    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:54.930486 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:54.930430    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:56.328220 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:56.328178    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:56.654151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:56.654043    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:57.790071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:57.790040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:59.194470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:59.194433    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:59.602884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:59.602837    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"]
Oct 30 20:53:59.672005 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:59.671970    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:53:59.773093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:53:59.773058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:00.287457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:00.287423    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:00.619690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:00.619471    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:01.606604 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:01.606569    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:02.375443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:02.375366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:02.526345 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:02.526294    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:02.751988 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:02.751875    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:02.920796 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:02.920760    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:03.335893 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:03.335857    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:03.476066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:03.476026    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:03.978092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:03.978058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:04.603843 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:04.603806    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd" podUID="604a5929-5208-4d37-84c8-49a113f22dd4" containerName="payload-processing" containerID="cri-o://da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca" gracePeriod=25
Oct 30 20:54:04.919301 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:04.919277    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:54:05.031091 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.031059    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/604a5929-5208-4d37-84c8-49a113f22dd4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/604a5929-5208-4d37-84c8-49a113f22dd4-plugins-config-volume\") pod \"604a5929-5208-4d37-84c8-49a113f22dd4\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") "
Oct 30 20:54:05.031596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.031203    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/604a5929-5208-4d37-84c8-49a113f22dd4-kube-api-access-kksvj\" (UniqueName: \"kubernetes.io/projected/604a5929-5208-4d37-84c8-49a113f22dd4-kube-api-access-kksvj\") pod \"604a5929-5208-4d37-84c8-49a113f22dd4\" (UID: \"604a5929-5208-4d37-84c8-49a113f22dd4\") "
Oct 30 20:54:05.032211 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.032186    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/604a5929-5208-4d37-84c8-49a113f22dd4-plugins-config-volume" pod "604a5929-5208-4d37-84c8-49a113f22dd4" (UID: "604a5929-5208-4d37-84c8-49a113f22dd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:54:05.034416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.034375    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/604a5929-5208-4d37-84c8-49a113f22dd4-kube-api-access-kksvj" pod "604a5929-5208-4d37-84c8-49a113f22dd4" (UID: "604a5929-5208-4d37-84c8-49a113f22dd4"). InnerVolumeSpecName "kube-api-access-kksvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:05.114858 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.114829    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:05.132928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.132890    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kksvj\" (UniqueName: \"kubernetes.io/projected/604a5929-5208-4d37-84c8-49a113f22dd4-kube-api-access-kksvj\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:05.132928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.132928    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/604a5929-5208-4d37-84c8-49a113f22dd4-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:05.548530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.548495    4852 generic.go:356] "Generic (PLEG): container finished" podID="604a5929-5208-4d37-84c8-49a113f22dd4" containerID="da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca" exitCode=0
Oct 30 20:54:05.548737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.548544    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"
Oct 30 20:54:05.548737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.548576    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd" event={"ID":"604a5929-5208-4d37-84c8-49a113f22dd4","Type":"ContainerDied","Data":"da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca"}
Oct 30 20:54:05.548737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.548640    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd" event={"ID":"604a5929-5208-4d37-84c8-49a113f22dd4","Type":"ContainerDied","Data":"1274dbf227958d21c1217eb0803310a7c541c20f62d9afef6304dd9dc4e89219"}
Oct 30 20:54:05.548737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.548671    4852 scope.go:122] "RemoveContainer" containerID="da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca"
Oct 30 20:54:05.584583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.584563    4852 scope.go:122] "RemoveContainer" containerID="da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca"
Oct 30 20:54:05.584927 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:05.584905    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca\": container with ID starting with da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca not found: ID does not exist" containerID="da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca"
Oct 30 20:54:05.584988 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.584938    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca"} err="failed to get container status \"da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca\": rpc error: code = NotFound desc = could not find container \"da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca\": container with ID starting with da172f6b6ba5b0933f6ad8a5c7b3116aae71a672bac7f8d86d8ed819f97a85ca not found: ID does not exist"
Oct 30 20:54:05.607157 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.607122    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"]
Oct 30 20:54:05.611549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.611523    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-95a684-7cb5598977-lkkjd"]
Oct 30 20:54:05.917968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:05.917886    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:06.006867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:06.006832    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="604a5929-5208-4d37-84c8-49a113f22dd4" path="/var/lib/kubelet/pods/604a5929-5208-4d37-84c8-49a113f22dd4/volumes"
Oct 30 20:54:06.739626 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:06.739591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:07.510331 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:07.510295    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:07.844133 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:07.844054    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"]
Oct 30 20:54:07.903818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:07.903785    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:08.002589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:08.002559    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:08.650941 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:08.650898    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:09.515513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:09.515467    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:09.830652 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:09.830578    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:09.935642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:09.935593    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:10.149494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.149408    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"]
Oct 30 20:54:10.150874 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.150814    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy" containerID="cri-o://44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542" gracePeriod=30
Oct 30 20:54:10.199353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.199321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:10.597312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.597282    4852 generic.go:356] "Generic (PLEG): container finished" podID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerID="81c0c9622ddc3c1c58cf0aca05ffce1e3ba5c4c229d4e9337e1927ba0aedfe6b" exitCode=0
Oct 30 20:54:10.597740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.597352    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" event={"ID":"499d2e0d-ebae-4f71-9e76-47f307049b15","Type":"ContainerDied","Data":"81c0c9622ddc3c1c58cf0aca05ffce1e3ba5c4c229d4e9337e1927ba0aedfe6b"}
Oct 30 20:54:10.597847 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.597832    4852 scope.go:122] "RemoveContainer" containerID="81c0c9622ddc3c1c58cf0aca05ffce1e3ba5c4c229d4e9337e1927ba0aedfe6b"
Oct 30 20:54:10.599485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.599461    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:54:10.599555 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.599527    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:54:10.599591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.599561    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:54:10.742679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:10.742617    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:11.607869 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:11.607836    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" event={"ID":"499d2e0d-ebae-4f71-9e76-47f307049b15","Type":"ContainerStarted","Data":"054e2258f485de37d6be18eea356f41b57f7ff09076c12bd83b9c3237eb3a544"}
Oct 30 20:54:11.814753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:11.814719    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:54:11.814936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:11.814902    4852 patch_prober.go:29] interesting pod/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 20:54:11.814989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:11.814970    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 20:54:11.875181 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:11.875078    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:12.643103 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:12.643062    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:12.814944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:12.814908    4852 patch_prober.go:29] interesting pod/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 20:54:12.815126 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:12.814964    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 20:54:12.844949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:12.844894    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756" podUID="bba7bd9a-e82b-4940-8dd5-9e244029a134" containerName="payload-pre-processing" containerID="cri-o://01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca" gracePeriod=25
Oct 30 20:54:13.024940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.024908    4852 scope.go:122] "RemoveContainer" containerID="295b69f13ee78dfe8f5f614de9d429d06ad6f87334f6fa13acb026607a40b36b"
Oct 30 20:54:13.195449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.195364    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:54:13.298117 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.298086    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bba7bd9a-e82b-4940-8dd5-9e244029a134-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba7bd9a-e82b-4940-8dd5-9e244029a134-plugins-config-volume\") pod \"bba7bd9a-e82b-4940-8dd5-9e244029a134\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") "
Oct 30 20:54:13.298326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.298283    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bba7bd9a-e82b-4940-8dd5-9e244029a134-kube-api-access-p2gcd\" (UniqueName: \"kubernetes.io/projected/bba7bd9a-e82b-4940-8dd5-9e244029a134-kube-api-access-p2gcd\") pod \"bba7bd9a-e82b-4940-8dd5-9e244029a134\" (UID: \"bba7bd9a-e82b-4940-8dd5-9e244029a134\") "
Oct 30 20:54:13.298554 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.298530    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba7bd9a-e82b-4940-8dd5-9e244029a134-plugins-config-volume" pod "bba7bd9a-e82b-4940-8dd5-9e244029a134" (UID: "bba7bd9a-e82b-4940-8dd5-9e244029a134"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:54:13.298875 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.298854    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba7bd9a-e82b-4940-8dd5-9e244029a134-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:13.300420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.300378    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba7bd9a-e82b-4940-8dd5-9e244029a134-kube-api-access-p2gcd" pod "bba7bd9a-e82b-4940-8dd5-9e244029a134" (UID: "bba7bd9a-e82b-4940-8dd5-9e244029a134"). InnerVolumeSpecName "kube-api-access-p2gcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:13.400572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.400535    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2gcd\" (UniqueName: \"kubernetes.io/projected/bba7bd9a-e82b-4940-8dd5-9e244029a134-kube-api-access-p2gcd\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:13.626043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.626009    4852 generic.go:356] "Generic (PLEG): container finished" podID="bba7bd9a-e82b-4940-8dd5-9e244029a134" containerID="01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca" exitCode=0
Oct 30 20:54:13.626246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.626063    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"
Oct 30 20:54:13.626246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.626089    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756" event={"ID":"bba7bd9a-e82b-4940-8dd5-9e244029a134","Type":"ContainerDied","Data":"01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca"}
Oct 30 20:54:13.626246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.626141    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756" event={"ID":"bba7bd9a-e82b-4940-8dd5-9e244029a134","Type":"ContainerDied","Data":"3e348a616c76e7535dc96817d88a9e7ea03da79c021379a25561f75558355454"}
Oct 30 20:54:13.626246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.626168    4852 scope.go:122] "RemoveContainer" containerID="01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca"
Oct 30 20:54:13.637799 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.637781    4852 scope.go:122] "RemoveContainer" containerID="01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca"
Oct 30 20:54:13.638104 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:13.638082    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca\": container with ID starting with 01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca not found: ID does not exist" containerID="01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca"
Oct 30 20:54:13.638214 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.638113    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca"} err="failed to get container status \"01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca\": rpc error: code = NotFound desc = could not find container \"01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca\": container with ID starting with 01d37fd72557928da7193f4c00e3f9835740ffeb732bc7929f06f6ea6a8cdaca not found: ID does not exist"
Oct 30 20:54:13.679146 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.679112    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"]
Oct 30 20:54:13.683538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.683510    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1982ff-75465cb8c5-nt756"]
Oct 30 20:54:13.714493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.714375    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:13.814863 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.814827    4852 patch_prober.go:29] interesting pod/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 20:54:13.815030 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:13.814899    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 20:54:14.006877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:14.006852    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bba7bd9a-e82b-4940-8dd5-9e244029a134" path="/var/lib/kubelet/pods/bba7bd9a-e82b-4940-8dd5-9e244029a134/volumes"
Oct 30 20:54:14.682444 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:14.682328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:14.814642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:14.814612    4852 patch_prober.go:29] interesting pod/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 20:54:14.814828 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:14.814678    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 20:54:15.496088 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.496053    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:54:15.503762 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.503620    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:15.534540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534512    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-data\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.534711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534553    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istiod-ca-cert\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.534711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534586    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-socket\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.534711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534616    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-envoy\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.534711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534644    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-podinfo\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.534938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534761    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-kube-api-access-nhbmm\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-kube-api-access-nhbmm\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.534938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534789    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-credential-socket\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.534938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534820    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-token\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-token\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.535081 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.534995    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-certs\") pod \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\" (UID: \"e59a1f96-18d7-48a9-85a9-fc5fd0027950\") "
Oct 30 20:54:15.535875 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.535851    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-certs" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:15.536331 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.536302    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-data" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:15.536536 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.536375    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-socket" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:15.536606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.536487    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istiod-ca-cert" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:54:15.536665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.536636    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-credential-socket" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:15.538349 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.538321    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-envoy" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:15.538520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.538454    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-kube-api-access-nhbmm" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "kube-api-access-nhbmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:15.538582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.538549    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-token" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:15.539002 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.538978    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-podinfo" pod "e59a1f96-18d7-48a9-85a9-fc5fd0027950" (UID: "e59a1f96-18d7-48a9-85a9-fc5fd0027950"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:54:15.637500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637462    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637502    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637518    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637534    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637547    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637558    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637574    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e59a1f96-18d7-48a9-85a9-fc5fd0027950-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637587    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhbmm\" (UniqueName: \"kubernetes.io/projected/e59a1f96-18d7-48a9-85a9-fc5fd0027950-kube-api-access-nhbmm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.637773 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.637598    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e59a1f96-18d7-48a9-85a9-fc5fd0027950-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:15.680500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.680378    4852 generic.go:356] "Generic (PLEG): container finished" podID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerID="44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542" exitCode=0
Oct 30 20:54:15.680736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.680714    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" event={"ID":"e59a1f96-18d7-48a9-85a9-fc5fd0027950","Type":"ContainerDied","Data":"44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542"}
Oct 30 20:54:15.680841 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.680790    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8" event={"ID":"e59a1f96-18d7-48a9-85a9-fc5fd0027950","Type":"ContainerDied","Data":"9b37e567dfa4ea80bd7e276570b5cf08ad7adea05793d5423b1e209585057816"}
Oct 30 20:54:15.680906 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.680870    4852 scope.go:122] "RemoveContainer" containerID="44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542"
Oct 30 20:54:15.681192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.681073    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"
Oct 30 20:54:15.694547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.694530    4852 scope.go:122] "RemoveContainer" containerID="44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542"
Oct 30 20:54:15.694859 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:15.694803    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542\": container with ID starting with 44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542 not found: ID does not exist" containerID="44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542"
Oct 30 20:54:15.694859 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.694832    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542"} err="failed to get container status \"44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542\": rpc error: code = NotFound desc = could not find container \"44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542\": container with ID starting with 44b03c25af9ed07df66a88a592eabbfe192a4b7002c385ecb29e6096ea174542 not found: ID does not exist"
Oct 30 20:54:15.709608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.709576    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"]
Oct 30 20:54:15.713887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.713863    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1fac16-openshift-default-cdc6dff7c-z8jx8"]
Oct 30 20:54:15.814680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.814638    4852 patch_prober.go:29] interesting pod/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 20:54:15.814867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:15.814704    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 20:54:16.006063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:16.006035    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" path="/var/lib/kubelet/pods/e59a1f96-18d7-48a9-85a9-fc5fd0027950/volumes"
Oct 30 20:54:16.025917 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:16.025885    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"]
Oct 30 20:54:16.030171 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:16.030140    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3df8e679-p5wdd"]
Oct 30 20:54:16.470466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:16.468912    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:16.814547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:16.814511    4852 patch_prober.go:29] interesting pod/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 20:54:16.814943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:16.814572    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 20:54:17.434364 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:17.434331    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:17.767202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:17.767165    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:54:17.820164 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:17.820135    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:54:18.005325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:18.005288    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="870d5026-8f4b-404f-9b4e-aa5164f35940" path="/var/lib/kubelet/pods/870d5026-8f4b-404f-9b4e-aa5164f35940/volumes"
Oct 30 20:54:18.542651 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:18.542617    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:18.705942 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:18.705912    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:54:19.421680 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:19.421639    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:20.173732 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:20.169254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:20.373408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:20.372894    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:20.491934 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:20.491842    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:20.751068 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:20.750960    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:21.021782 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:21.021745    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"]
Oct 30 20:54:21.092609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:21.092577    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:21.194113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:21.194082    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:21.561264 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:21.561228    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:54:22.116748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:22.116718    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:23.061517 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:23.061487    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:23.690985 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:23.690950    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"]
Oct 30 20:54:23.691256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:23.691226    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8" podUID="ff3d46f4-7d9d-4166-95de-23f794b86212" containerName="maas-api" containerID="cri-o://cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14" gracePeriod=30
Oct 30 20:54:23.732441 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:23.732413    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:23.831670 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:23.831635    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:23.963463 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:23.945727    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:24.138982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.138950    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:54:24.221997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.221918    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff3d46f4-7d9d-4166-95de-23f794b86212-kube-api-access-j9mbv\" (UniqueName: \"kubernetes.io/projected/ff3d46f4-7d9d-4166-95de-23f794b86212-kube-api-access-j9mbv\") pod \"ff3d46f4-7d9d-4166-95de-23f794b86212\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") "
Oct 30 20:54:24.221997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.221971    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-metrics-certs\") pod \"ff3d46f4-7d9d-4166-95de-23f794b86212\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") "
Oct 30 20:54:24.222233 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.222130    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff3d46f4-7d9d-4166-95de-23f794b86212-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3d46f4-7d9d-4166-95de-23f794b86212-openshift-service-ca\") pod \"ff3d46f4-7d9d-4166-95de-23f794b86212\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") "
Oct 30 20:54:24.222359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.222294    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-maas-api-tls\") pod \"ff3d46f4-7d9d-4166-95de-23f794b86212\" (UID: \"ff3d46f4-7d9d-4166-95de-23f794b86212\") "
Oct 30 20:54:24.222690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.222651    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3d46f4-7d9d-4166-95de-23f794b86212-openshift-service-ca" pod "ff3d46f4-7d9d-4166-95de-23f794b86212" (UID: "ff3d46f4-7d9d-4166-95de-23f794b86212"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:54:24.223048 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.223027    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3d46f4-7d9d-4166-95de-23f794b86212-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:24.225011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.224984    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-metrics-certs" pod "ff3d46f4-7d9d-4166-95de-23f794b86212" (UID: "ff3d46f4-7d9d-4166-95de-23f794b86212"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:54:24.225219 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.225198    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-maas-api-tls" pod "ff3d46f4-7d9d-4166-95de-23f794b86212" (UID: "ff3d46f4-7d9d-4166-95de-23f794b86212"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:54:24.225471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.225445    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff3d46f4-7d9d-4166-95de-23f794b86212-kube-api-access-j9mbv" pod "ff3d46f4-7d9d-4166-95de-23f794b86212" (UID: "ff3d46f4-7d9d-4166-95de-23f794b86212"). InnerVolumeSpecName "kube-api-access-j9mbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:24.229631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.229604    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:24.304319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.304279    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:24.324527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.324492    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9mbv\" (UniqueName: \"kubernetes.io/projected/ff3d46f4-7d9d-4166-95de-23f794b86212-kube-api-access-j9mbv\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:24.324527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.324524    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:24.324738 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.324537    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff3d46f4-7d9d-4166-95de-23f794b86212-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:24.719665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.719628    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:24.757398 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.757352    4852 generic.go:356] "Generic (PLEG): container finished" podID="ff3d46f4-7d9d-4166-95de-23f794b86212" containerID="cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14" exitCode=0
Oct 30 20:54:24.757561 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.757444    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"
Oct 30 20:54:24.757561 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.757441    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8" event={"ID":"ff3d46f4-7d9d-4166-95de-23f794b86212","Type":"ContainerDied","Data":"cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14"}
Oct 30 20:54:24.757561 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.757545    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8" event={"ID":"ff3d46f4-7d9d-4166-95de-23f794b86212","Type":"ContainerDied","Data":"93e95606fe7201b45522de32def15315a060b772582acc2a0d500d442d71add6"}
Oct 30 20:54:24.757671 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.757576    4852 scope.go:122] "RemoveContainer" containerID="cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14"
Oct 30 20:54:24.770165 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.770139    4852 scope.go:122] "RemoveContainer" containerID="cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14"
Oct 30 20:54:24.770531 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:24.770505    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14\": container with ID starting with cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14 not found: ID does not exist" containerID="cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14"
Oct 30 20:54:24.770630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.770538    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14"} err="failed to get container status \"cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14\": rpc error: code = NotFound desc = could not find container \"cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14\": container with ID starting with cbad2c508b0d877677b0257d041f53f79706f4a107941ffda245d32f62a07a14 not found: ID does not exist"
Oct 30 20:54:24.784678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.784649    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"]
Oct 30 20:54:24.788890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:24.788870    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1982ff-d679d459f-mkbr8"]
Oct 30 20:54:25.234874 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:25.234839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:26.006719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.006668    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff3d46f4-7d9d-4166-95de-23f794b86212" path="/var/lib/kubelet/pods/ff3d46f4-7d9d-4166-95de-23f794b86212/volumes"
Oct 30 20:54:26.022688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.022645    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92" podUID="db620f75-26b3-4049-aaa7-20e403e1c753" containerName="payload-processing" containerID="cri-o://0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0" gracePeriod=25
Oct 30 20:54:26.178431 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.178381    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:26.279714 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.279677    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:26.357065 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.357027    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:54:26.500912 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.500876    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db620f75-26b3-4049-aaa7-20e403e1c753-kube-api-access-8g89l\" (UniqueName: \"kubernetes.io/projected/db620f75-26b3-4049-aaa7-20e403e1c753-kube-api-access-8g89l\") pod \"db620f75-26b3-4049-aaa7-20e403e1c753\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") "
Oct 30 20:54:26.501143 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.501124    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db620f75-26b3-4049-aaa7-20e403e1c753-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db620f75-26b3-4049-aaa7-20e403e1c753-plugins-config-volume\") pod \"db620f75-26b3-4049-aaa7-20e403e1c753\" (UID: \"db620f75-26b3-4049-aaa7-20e403e1c753\") "
Oct 30 20:54:26.502217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.502188    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db620f75-26b3-4049-aaa7-20e403e1c753-plugins-config-volume" pod "db620f75-26b3-4049-aaa7-20e403e1c753" (UID: "db620f75-26b3-4049-aaa7-20e403e1c753"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:54:26.509476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.509447    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db620f75-26b3-4049-aaa7-20e403e1c753-kube-api-access-8g89l" pod "db620f75-26b3-4049-aaa7-20e403e1c753" (UID: "db620f75-26b3-4049-aaa7-20e403e1c753"). InnerVolumeSpecName "kube-api-access-8g89l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:26.603359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.603271    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g89l\" (UniqueName: \"kubernetes.io/projected/db620f75-26b3-4049-aaa7-20e403e1c753-kube-api-access-8g89l\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:26.603359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.603306    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db620f75-26b3-4049-aaa7-20e403e1c753-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:26.666024 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.665990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:26.778564 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.778533    4852 generic.go:356] "Generic (PLEG): container finished" podID="db620f75-26b3-4049-aaa7-20e403e1c753" containerID="0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0" exitCode=0
Oct 30 20:54:26.778760 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.778579    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"
Oct 30 20:54:26.778760 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.778624    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92" event={"ID":"db620f75-26b3-4049-aaa7-20e403e1c753","Type":"ContainerDied","Data":"0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0"}
Oct 30 20:54:26.778760 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.778721    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92" event={"ID":"db620f75-26b3-4049-aaa7-20e403e1c753","Type":"ContainerDied","Data":"64cc3759d04bf8f7df169adfdee6426dc9a10b54fde4b4f5ae420773bfe48c66"}
Oct 30 20:54:26.778890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.778767    4852 scope.go:122] "RemoveContainer" containerID="0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0"
Oct 30 20:54:26.790113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.790096    4852 scope.go:122] "RemoveContainer" containerID="0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0"
Oct 30 20:54:26.790409 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:26.790376    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0\": container with ID starting with 0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0 not found: ID does not exist" containerID="0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0"
Oct 30 20:54:26.790469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.790420    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0"} err="failed to get container status \"0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0\": rpc error: code = NotFound desc = could not find container \"0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0\": container with ID starting with 0ad7d2cc182a2cf3131b48b28320c870b02b113dcad2e6255e9b27d9707a43e0 not found: ID does not exist"
Oct 30 20:54:26.804364 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.804334    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"]
Oct 30 20:54:26.811499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:26.811474    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-49990a-57945d5ccc-6bl92"]
Oct 30 20:54:27.574951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:27.574897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:28.008864 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:28.008835    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db620f75-26b3-4049-aaa7-20e403e1c753" path="/var/lib/kubelet/pods/db620f75-26b3-4049-aaa7-20e403e1c753/volumes"
Oct 30 20:54:28.285774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:28.285672    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:28.389312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:28.389279    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:28.486237 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:28.486201    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:28.736903 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:28.736820    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"]
Oct 30 20:54:28.737311 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:28.737108    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" containerID="cri-o://054e2258f485de37d6be18eea356f41b57f7ff09076c12bd83b9c3237eb3a544" gracePeriod=30
Oct 30 20:54:28.746292 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:28.746263    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:29.057867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:29.057799    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"]
Oct 30 20:54:29.065518 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:29.065482    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b91eb345-nhc46"]
Oct 30 20:54:29.966606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:29.966570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:30.011676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:30.011644    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2415084f-ae27-4510-83e0-5c84adca4ad6" path="/var/lib/kubelet/pods/2415084f-ae27-4510-83e0-5c84adca4ad6/volumes"
Oct 30 20:54:30.916368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:30.916336    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:31.842515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:31.842474    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:32.222842 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:32.222746    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:32.424776 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:32.424724    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:32.525476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:32.525164    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:32.629919 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:32.629871    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:32.756529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:32.756488    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:32.880457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:32.879712    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:32.965473 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:32.965435    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:33.108355 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:33.108318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:33.444972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:33.444938    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:33.707456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:33.707342    4852 patch_prober.go:29] interesting pod/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 20:54:33.707456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:33.707428    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:54:33.857148 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:33.857121    4852 generic.go:356] "Generic (PLEG): container finished" podID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerID="054e2258f485de37d6be18eea356f41b57f7ff09076c12bd83b9c3237eb3a544" exitCode=0
Oct 30 20:54:33.857245 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:33.857191    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" event={"ID":"499d2e0d-ebae-4f71-9e76-47f307049b15","Type":"ContainerDied","Data":"054e2258f485de37d6be18eea356f41b57f7ff09076c12bd83b9c3237eb3a544"}
Oct 30 20:54:33.857310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:33.857257    4852 scope.go:122] "RemoveContainer" containerID="81c0c9622ddc3c1c58cf0aca05ffce1e3ba5c4c229d4e9337e1927ba0aedfe6b"
Oct 30 20:54:34.036402 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.036361    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:54:34.185582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185552    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-certs\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.185783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185641    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-token\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-token\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.185783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185772    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-data\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.185885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185801    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-podinfo\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.185885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185837    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-certs" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:34.185885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185864    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/499d2e0d-ebae-4f71-9e76-47f307049b15-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499d2e0d-ebae-4f71-9e76-47f307049b15-istiod-ca-cert\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.186016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185917    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-envoy\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.186016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.185954    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-kube-api-access-dnnr5\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-kube-api-access-dnnr5\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.186016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.186005    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-credential-socket\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.186176 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.186031    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-socket\") pod \"499d2e0d-ebae-4f71-9e76-47f307049b15\" (UID: \"499d2e0d-ebae-4f71-9e76-47f307049b15\") "
Oct 30 20:54:34.186176 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.186053    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-data" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:34.186312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.186284    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-credential-socket" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:34.186510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.186232    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/499d2e0d-ebae-4f71-9e76-47f307049b15-istiod-ca-cert" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:54:34.186618 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.186541    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-socket" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:34.187073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.186983    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.187170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.187156    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.187243 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.187182    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.187243 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.187197    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.187243 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.187234    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499d2e0d-ebae-4f71-9e76-47f307049b15-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.188216 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.188193    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-envoy" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:34.188469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.188445    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-podinfo" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:54:34.188642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.188620    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-kube-api-access-dnnr5" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "kube-api-access-dnnr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:34.189108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.189093    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-token" pod "499d2e0d-ebae-4f71-9e76-47f307049b15" (UID: "499d2e0d-ebae-4f71-9e76-47f307049b15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:34.289183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.289099    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.289183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.289130    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.289183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.289140    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499d2e0d-ebae-4f71-9e76-47f307049b15-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.289183 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.289149    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnnr5\" (UniqueName: \"kubernetes.io/projected/499d2e0d-ebae-4f71-9e76-47f307049b15-kube-api-access-dnnr5\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:34.421787 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.421753    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:34.867339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.867303    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj" event={"ID":"499d2e0d-ebae-4f71-9e76-47f307049b15","Type":"ContainerDied","Data":"e8ce8247dac94bc6df180b89b69b5eadb2074b42f89570a4f9e71aeae9271586"}
Oct 30 20:54:34.867339 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.867327    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"
Oct 30 20:54:34.867624 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.867368    4852 scope.go:122] "RemoveContainer" containerID="054e2258f485de37d6be18eea356f41b57f7ff09076c12bd83b9c3237eb3a544"
Oct 30 20:54:34.893751 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.893724    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"]
Oct 30 20:54:34.899033 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:34.899009    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-95a684-openshift-default-7dd56b5cf9-sj9pj"]
Oct 30 20:54:35.233542 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:35.233453    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:36.004970 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:36.004938    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" path="/var/lib/kubelet/pods/499d2e0d-ebae-4f71-9e76-47f307049b15/volumes"
Oct 30 20:54:36.065588 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:36.065553    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:37.035596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:37.035562    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:37.753312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:37.753281    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:38.596285 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:38.596249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:39.372665 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:39.372624    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:39.458831 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:39.458775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:40.261144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:40.261109    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:40.848306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:40.848270    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:40.949983 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:40.949934    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:41.130428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:41.130273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:41.266736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:41.266704    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"]
Oct 30 20:54:41.267170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:41.266951    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy" containerID="cri-o://df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845" gracePeriod=30
Oct 30 20:54:41.281728 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:41.281701    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:42.046748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:42.046717    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:42.841325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:42.841290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:43.664620 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:43.664379    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:43.763050 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:43.763016    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:45.201038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.200993    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.216426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.216140    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:45.323422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.321500    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 20:54:45.408524 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.408467    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 20:54:45.436570 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.436505    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.456602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.456488    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 20:54:45.495095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.494505    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.521943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.521897    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.553413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.549233    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.612610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.612544    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.658411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.653673    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" podUID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.100:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.658411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.654075    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 20:54:45.787571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.772820    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": context deadline exceeded"
Oct 30 20:54:45.787571 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.773361    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:54:45.875440 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:45.835187    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 20:54:46.224296 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.224208    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:46.573126 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.573098    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:54:46.616955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.616855    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-envoy\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.616955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.616952    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-token\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-token\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.617202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.617034    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-data\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.617202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.617082    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-socket\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.617202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.617177    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-certs\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.617420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.617312    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istiod-ca-cert\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.617420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.617352    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-credential-socket\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.617527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.617508    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-podinfo\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.617587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.617571    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-kube-api-access-spfb5\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-kube-api-access-spfb5\") pod \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\" (UID: \"591a15d1-0222-4ea1-b512-7c4ac5d726c2\") "
Oct 30 20:54:46.618822 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.618791    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-certs" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:46.618939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.618815    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-data" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:46.619044 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.619021    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-socket" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:46.619101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.619042    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-credential-socket" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:46.619364 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.619341    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istiod-ca-cert" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:54:46.619853 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.619825    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-envoy" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:54:46.621145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.621118    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-token" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:46.621532 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.621508    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-kube-api-access-spfb5" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "kube-api-access-spfb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:54:46.621612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.621571    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-podinfo" pod "591a15d1-0222-4ea1-b512-7c4ac5d726c2" (UID: "591a15d1-0222-4ea1-b512-7c4ac5d726c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:54:46.719509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719474    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719505    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spfb5\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-kube-api-access-spfb5\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719516    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719525    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719534    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719542    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719551    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719559    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/591a15d1-0222-4ea1-b512-7c4ac5d726c2-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:46.719757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:46.719568    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/591a15d1-0222-4ea1-b512-7c4ac5d726c2-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:54:47.062809 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.062772    4852 generic.go:356] "Generic (PLEG): container finished" podID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerID="df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845" exitCode=0
Oct 30 20:54:47.063006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.062834    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"
Oct 30 20:54:47.063006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.062848    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" event={"ID":"591a15d1-0222-4ea1-b512-7c4ac5d726c2","Type":"ContainerDied","Data":"df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845"}
Oct 30 20:54:47.063006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.062901    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r" event={"ID":"591a15d1-0222-4ea1-b512-7c4ac5d726c2","Type":"ContainerDied","Data":"4676d7ca223baf06d31a5c031fd0e502341d3309ededb60e4fc92db52ee08751"}
Oct 30 20:54:47.063006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.062925    4852 scope.go:122] "RemoveContainer" containerID="df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845"
Oct 30 20:54:47.074625 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.074606    4852 scope.go:122] "RemoveContainer" containerID="287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c"
Oct 30 20:54:47.084714 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.084693    4852 scope.go:122] "RemoveContainer" containerID="df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845"
Oct 30 20:54:47.085063 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:47.085027    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845\": container with ID starting with df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845 not found: ID does not exist" containerID="df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845"
Oct 30 20:54:47.085151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.085066    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845"} err="failed to get container status \"df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845\": rpc error: code = NotFound desc = could not find container \"df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845\": container with ID starting with df20092bae46d5abd04f9fea44e5e024867a767ad98fe4f49982fb0f490a6845 not found: ID does not exist"
Oct 30 20:54:47.085151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.085090    4852 scope.go:122] "RemoveContainer" containerID="287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c"
Oct 30 20:54:47.085449 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:47.085424    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c\": container with ID starting with 287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c not found: ID does not exist" containerID="287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c"
Oct 30 20:54:47.085516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.085456    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c"} err="failed to get container status \"287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c\": rpc error: code = NotFound desc = could not find container \"287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c\": container with ID starting with 287070ee07140e55da7c59745ddb73c9a7c14edd8a3af4c7c144f8393b95449c not found: ID does not exist"
Oct 30 20:54:47.105448 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.105413    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"]
Oct 30 20:54:47.109423 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.109373    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-49990a-openshift-default-6bdfd54755-f8c2r"]
Oct 30 20:54:47.149556 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.149495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:47.918467 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:47.918429    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:48.006260 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:48.006223    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" path="/var/lib/kubelet/pods/591a15d1-0222-4ea1-b512-7c4ac5d726c2/volumes"
Oct 30 20:54:48.679238 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:48.679205    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:49.455707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:49.455671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:49.713884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:49.713797    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:49.905989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:49.905959    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:50.068508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:50.068469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:51.076577 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:51.076537    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:51.561056 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:54:51.561022    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:54:52.021881 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:52.021848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:53.012482 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:53.012427    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:53.769165 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:53.769120    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:53.959520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:53.959483    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:54.717673 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:54.717634    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:55.445660 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:55.445629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:55.620091 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:55.620059    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:55.717207 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:55.717068    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:55.954452 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:55.954137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:56.153113 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:56.153077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:56.239130 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:56.239098    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:56.320622 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:56.320554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:56.702056 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:56.702002    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:57.236879 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:57.236848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:58.086949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:58.086904    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:58.923479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:58.923427    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:54:59.839373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:54:59.839337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:00.483006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:00.482953    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:00.585080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:00.585047    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:00.728670 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:00.728628    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:00.871703 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:00.871666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:00.969702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:00.969658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:01.864291 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:01.864261    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:02.854835 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:02.854806    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:03.543294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:03.543259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:03.651433 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:03.651371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:03.815508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:03.815379    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:04.068057 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:04.067974    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:04.256405 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:04.253049    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:04.370668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:04.370575    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:05.031623 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:05.031577    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:06.318413 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:06.318365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:06.412029 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:06.411998    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:07.034118 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:07.034083    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:07.137613 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:07.137558    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:07.237092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:07.237061    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:07.525242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:07.525202    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:07.637655 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:07.637607    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:07.746847 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:07.746812    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:07.840738 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:07.840633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:08.373358 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:08.373327    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:09.410003 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:09.409969    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:10.333116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:10.333072    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:10.885225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:10.885192    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:11.019967 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:11.019932    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:11.114778 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:11.114746    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:11.450398 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:11.450340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:12.027315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:12.027277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:12.119275 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:12.119201    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:12.736465 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:12.736372    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:12.998274 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:12.998188    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:13.093913 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:13.093880    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:13.107445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:13.107382    4852 scope.go:122] "RemoveContainer" containerID="78db8d6dce4cdcd77d8791e0d733592e0d2e59706644943cddbc0099097b9207"
Oct 30 20:55:13.725806 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:13.725744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:14.525064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:14.525028    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:15.303597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:15.303561    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:16.121256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:16.121223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:16.828651 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:16.828621    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:16.923871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:16.923840    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:18.021662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:18.021632    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:18.118304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:18.118275    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:18.218426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:18.218359    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:18.941331 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:18.941296    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:19.715437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:19.715403    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:20.429165 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:20.429135    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:20.734422 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:20.734312    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:21.251602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:21.251571    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:21.355857 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:21.355823    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:21.568722 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:55:21.568678    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:55:21.647647 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:21.647610    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:22.466373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:22.466342    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:23.426512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:23.426476    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:24.221518 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:24.221485    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:24.989243 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:24.989210    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:25.089191 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:25.089156    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:25.308453 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:25.308419    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:26.208304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:26.208269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:26.321398 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:26.321354    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:28.121485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:28.121445    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:29.246548 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:29.246513    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:30.287437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:30.287382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:30.981523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:30.981480    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:31.079890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:31.079858    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:31.223242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:31.223201    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:31.317692 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:31.317658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:31.409911 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:31.409875    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:31.761319 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:31.761276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:32.811727 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:32.811678    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:33.739320 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:33.739282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:34.543239 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.543204    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"]
Oct 30 20:55:34.543974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.543951    4852 cpu_manager.go:414] "containerMap: removing container" podUID="604a5929-5208-4d37-84c8-49a113f22dd4" containerName="payload-processing"
Oct 30 20:55:34.543974 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.543973    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="604a5929-5208-4d37-84c8-49a113f22dd4" containerName="payload-processing"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.543984    4852 cpu_manager.go:414] "containerMap: removing container" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.543989    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.543999    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ff3d46f4-7d9d-4166-95de-23f794b86212" containerName="maas-api"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544004    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff3d46f4-7d9d-4166-95de-23f794b86212" containerName="maas-api"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544016    4852 cpu_manager.go:414] "containerMap: removing container" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544020    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544037    4852 cpu_manager.go:414] "containerMap: removing container" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544041    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544050    4852 cpu_manager.go:414] "containerMap: removing container" podUID="bba7bd9a-e82b-4940-8dd5-9e244029a134" containerName="payload-pre-processing"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544055    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba7bd9a-e82b-4940-8dd5-9e244029a134" containerName="payload-pre-processing"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544062    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544066    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544080    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d7407621-98f1-44a3-8e23-2e7cffc888ec" containerName="payload-pre-processing"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544086    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7407621-98f1-44a3-8e23-2e7cffc888ec" containerName="payload-pre-processing"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544094    4852 cpu_manager.go:414] "containerMap: removing container" podUID="db620f75-26b3-4049-aaa7-20e403e1c753" containerName="payload-processing"
Oct 30 20:55:34.544161 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544098    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db620f75-26b3-4049-aaa7-20e403e1c753" containerName="payload-processing"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544203    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="604a5929-5208-4d37-84c8-49a113f22dd4" containerName="payload-processing"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544215    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544224    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff3d46f4-7d9d-4166-95de-23f794b86212" containerName="maas-api"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544233    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544245    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba7bd9a-e82b-4940-8dd5-9e244029a134" containerName="payload-pre-processing"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544252    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e59a1f96-18d7-48a9-85a9-fc5fd0027950" containerName="istio-proxy"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544265    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7407621-98f1-44a3-8e23-2e7cffc888ec" containerName="payload-pre-processing"
Oct 30 20:55:34.544959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.544273    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="db620f75-26b3-4049-aaa7-20e403e1c753" containerName="payload-processing"
Oct 30 20:55:34.548059 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.548040    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.561639 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.556879    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-7c14e2-serving-cert\""
Oct 30 20:55:34.569240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.569209    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"]
Oct 30 20:55:34.591445 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.591360    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:34.692016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.691981    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:34.697504 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.697477    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4232a13e-0d80-4c2f-9497-0d5c944369a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.697674 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.697652    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rctf7\" (UniqueName: \"kubernetes.io/projected/4232a13e-0d80-4c2f-9497-0d5c944369a1-kube-api-access-rctf7\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.697734 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.697721    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-metrics-certs\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.697792 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.697778    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.792799 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.792759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:34.800010 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.799919    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-metrics-certs\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.806072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.806036    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.806266 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.806236    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4232a13e-0d80-4c2f-9497-0d5c944369a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.806477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.806456    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rctf7\" (UniqueName: \"kubernetes.io/projected/4232a13e-0d80-4c2f-9497-0d5c944369a1-kube-api-access-rctf7\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.807195 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.807158    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4232a13e-0d80-4c2f-9497-0d5c944369a1-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.808986 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.808961    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.811122 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.811098    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-metrics-certs\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.818093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.818063    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rctf7\" (UniqueName: \"kubernetes.io/projected/4232a13e-0d80-4c2f-9497-0d5c944369a1-kube-api-access-rctf7\") pod \"maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:34.872278 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:34.872243    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:35.156217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:35.156174    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"]
Oct 30 20:55:35.156342 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:55:35.156174    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4232a13e_0d80_4c2f_9497_0d5c944369a1.slice/crio-c0cb5ec463f928af5c07797d5dc9d639c4eb2401fae148858258f65bccf28b4a WatchSource:0}: Error finding container c0cb5ec463f928af5c07797d5dc9d639c4eb2401fae148858258f65bccf28b4a: Status 404 returned error can't find the container with id c0cb5ec463f928af5c07797d5dc9d639c4eb2401fae148858258f65bccf28b4a
Oct 30 20:55:35.598417 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:35.598375    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:35.636378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:35.636340    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d" event={"ID":"4232a13e-0d80-4c2f-9497-0d5c944369a1","Type":"ContainerStarted","Data":"c0cb5ec463f928af5c07797d5dc9d639c4eb2401fae148858258f65bccf28b4a"}
Oct 30 20:55:36.647462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:36.647423    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d" event={"ID":"4232a13e-0d80-4c2f-9497-0d5c944369a1","Type":"ContainerStarted","Data":"552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe"}
Oct 30 20:55:36.647866 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:36.647495    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:36.673315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:36.673250    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d" podStartSLOduration=2.262561904 podStartE2EDuration="2.673233104s" totalImagesPullingTime="410.6712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:55:36.672754029 +0000 UTC m=+1044.787488219" watchObservedRunningTime="2026-10-30 20:55:36.673233104 +0000 UTC m=+1044.787967296"
Oct 30 20:55:36.697194 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:36.697160    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:37.233529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:37.233497    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:37.329975 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:37.329939    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:37.957175 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:37.957131    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:38.496291 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:38.496256    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:38.591875 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:38.591839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:39.209074 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:39.209040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:39.931861 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:39.931829    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:40.376323 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:40.376286    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:40.476509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:40.476463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:40.887097 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:40.887063    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:41.535487 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:41.535451    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:41.711835 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:41.711779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:41.839466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:41.839242    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:42.209594 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:42.209508    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:42.435063 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:42.435031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:42.664355 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:42.664329    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:55:43.309982 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:43.309926    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:44.306068 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:44.306034    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:45.545910 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:45.545878    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:46.785980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:46.785946    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:47.407993 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:47.407956    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"]
Oct 30 20:55:47.466064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:47.466029    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:47.969767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:47.969729    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:49.355858 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:49.355816    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:50.228669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:50.228636    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:50.330872 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:50.330839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:50.491076 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:50.490980    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:51.228058 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:51.228024    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:51.558674 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:55:51.558629    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:55:52.121091 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:52.121060    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:53.057370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:53.057340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:53.881600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:53.881566    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:54.297547 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:54.297512    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:54.622979 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:54.622900    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:54.729836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:54.729799    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:54.858496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:54.858445    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:55.527740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:55.527705    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:56.028059 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:56.028029    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"]
Oct 30 20:55:56.031838 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:56.031817    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fb76500-tgqhs"]
Oct 30 20:55:56.422740 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:56.422652    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:57.340550 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:57.340507    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:57.840632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:57.840603    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:57.940664 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:57.940629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:58.005943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.005904    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec91dca4-c220-48e7-af17-a5c1780eabda" path="/var/lib/kubelet/pods/ec91dca4-c220-48e7-af17-a5c1780eabda/volumes"
Oct 30 20:55:58.299062 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.299025    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:58.401805 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.401769    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:58.497367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.497332    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:58.598120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.598011    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:58.701850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.701793    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:58.829699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.829664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:58.931370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:58.931281    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:55:59.704817 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:55:59.704768    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:00.187826 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.187792    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:00.241343 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.241310    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"]
Oct 30 20:56:00.242273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.242233    4852 cpu_manager.go:414] "containerMap: removing container" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy"
Oct 30 20:56:00.242273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.242259    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy"
Oct 30 20:56:00.242522 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.242503    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="591a15d1-0222-4ea1-b512-7c4ac5d726c2" containerName="istio-proxy"
Oct 30 20:56:00.242589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.242539    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="499d2e0d-ebae-4f71-9e76-47f307049b15" containerName="istio-proxy"
Oct 30 20:56:00.247021 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.246989    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.256743 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.256717    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-319188-dockercfg-49hr6\""
Oct 30 20:56:00.257619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.257590    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"]
Oct 30 20:56:00.293700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.293672    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:00.337084 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.337055    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-319188\""
Oct 30 20:56:00.362646 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.362608    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8rhn\" (UniqueName: \"kubernetes.io/projected/a117995e-6ced-4d79-b4be-2e7604d0a8d6-kube-api-access-k8rhn\") pod \"payload-processing-e2e-api-a-319188-59f459b575-r26vz\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.362812 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.362788    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a117995e-6ced-4d79-b4be-2e7604d0a8d6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-319188-59f459b575-r26vz\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.467201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.467100    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8rhn\" (UniqueName: \"kubernetes.io/projected/a117995e-6ced-4d79-b4be-2e7604d0a8d6-kube-api-access-k8rhn\") pod \"payload-processing-e2e-api-a-319188-59f459b575-r26vz\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.467610 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.467580    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a117995e-6ced-4d79-b4be-2e7604d0a8d6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-319188-59f459b575-r26vz\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.468733 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.468694    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a117995e-6ced-4d79-b4be-2e7604d0a8d6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-319188-59f459b575-r26vz\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.497500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.480144    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8rhn\" (UniqueName: \"kubernetes.io/projected/a117995e-6ced-4d79-b4be-2e7604d0a8d6-kube-api-access-k8rhn\") pod \"payload-processing-e2e-api-a-319188-59f459b575-r26vz\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.579682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.579649    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:00.913277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:00.913242    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"]
Oct 30 20:56:00.918084 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:56:00.918052    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda117995e_6ced_4d79_b4be_2e7604d0a8d6.slice/crio-134b0200f4204ea8cc842fb62e5a060868d1d0ba56852eda0d949a797f7a715a WatchSource:0}: Error finding container 134b0200f4204ea8cc842fb62e5a060868d1d0ba56852eda0d949a797f7a715a: Status 404 returned error can't find the container with id 134b0200f4204ea8cc842fb62e5a060868d1d0ba56852eda0d949a797f7a715a
Oct 30 20:56:01.235332 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:01.235250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:01.909560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:01.909522    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz" event={"ID":"a117995e-6ced-4d79-b4be-2e7604d0a8d6","Type":"ContainerStarted","Data":"5af1332e1e01e54a9feb0c4053cd79a37a987d83b25049c5f0541f2afdb8d524"}
Oct 30 20:56:01.909760 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:01.909579    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz" event={"ID":"a117995e-6ced-4d79-b4be-2e7604d0a8d6","Type":"ContainerStarted","Data":"134b0200f4204ea8cc842fb62e5a060868d1d0ba56852eda0d949a797f7a715a"}
Oct 30 20:56:01.909760 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:01.909631    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:01.932705 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:01.932631    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz" podStartSLOduration=1.932611186 podStartE2EDuration="1.932611186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:56:01.926304994 +0000 UTC m=+1070.041039184" watchObservedRunningTime="2026-10-30 20:56:01.932611186 +0000 UTC m=+1070.047345377"
Oct 30 20:56:02.024912 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.024853    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:02.126377 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.126337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:02.225480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.225380    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:02.336395 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.336358    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:02.430943 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.430892    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" podUID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" containerName="main" containerID="cri-o://2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04" gracePeriod=45
Oct 30 20:56:02.440991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.440964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:02.535639 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.535603    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:02.731642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.731619    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:56:02.812135 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812043    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-home\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-home\") pod \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") "
Oct 30 20:56:02.812286 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812183    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tmp-dir\") pod \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") "
Oct 30 20:56:02.812354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812305    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-model-cache\") pod \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") "
Oct 30 20:56:02.812354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812337    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-home" pod "6f74e3ec-1cc6-4223-9783-bef1f4fea17f" (UID: "6f74e3ec-1cc6-4223-9783-bef1f4fea17f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:02.812495 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812356    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-dshm\") pod \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") "
Oct 30 20:56:02.812552 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812512    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tls-certs\") pod \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") "
Oct 30 20:56:02.812552 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812509    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tmp-dir" pod "6f74e3ec-1cc6-4223-9783-bef1f4fea17f" (UID: "6f74e3ec-1cc6-4223-9783-bef1f4fea17f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:02.812662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812550    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-kube-api-access-ch6jx\" (UniqueName: \"kubernetes.io/projected/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-kube-api-access-ch6jx\") pod \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\" (UID: \"6f74e3ec-1cc6-4223-9783-bef1f4fea17f\") "
Oct 30 20:56:02.812662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.812564    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-model-cache" pod "6f74e3ec-1cc6-4223-9783-bef1f4fea17f" (UID: "6f74e3ec-1cc6-4223-9783-bef1f4fea17f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:02.813232 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.813206    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:02.813358 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.813235    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:02.813358 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.813249    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:02.815260 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.815234    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-dshm" pod "6f74e3ec-1cc6-4223-9783-bef1f4fea17f" (UID: "6f74e3ec-1cc6-4223-9783-bef1f4fea17f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:02.815513 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.815459    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-kube-api-access-ch6jx" pod "6f74e3ec-1cc6-4223-9783-bef1f4fea17f" (UID: "6f74e3ec-1cc6-4223-9783-bef1f4fea17f"). InnerVolumeSpecName "kube-api-access-ch6jx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:56:02.824829 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.824801    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tls-certs" pod "6f74e3ec-1cc6-4223-9783-bef1f4fea17f" (UID: "6f74e3ec-1cc6-4223-9783-bef1f4fea17f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:56:02.914820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.914783    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:02.914820 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.914822    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch6jx\" (UniqueName: \"kubernetes.io/projected/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-kube-api-access-ch6jx\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:02.915066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.914840    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f74e3ec-1cc6-4223-9783-bef1f4fea17f-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:02.918975 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.918947    4852 generic.go:356] "Generic (PLEG): container finished" podID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" containerID="2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04" exitCode=0
Oct 30 20:56:02.919101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.918995    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"
Oct 30 20:56:02.919101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.919034    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" event={"ID":"6f74e3ec-1cc6-4223-9783-bef1f4fea17f","Type":"ContainerDied","Data":"2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04"}
Oct 30 20:56:02.919101 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.919083    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" event={"ID":"6f74e3ec-1cc6-4223-9783-bef1f4fea17f","Type":"ContainerDied","Data":"b6b0bd4594409c6e213ce2129023d57030285870e4e0e2ab615ac5fe538526e4"}
Oct 30 20:56:02.919217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.919100    4852 scope.go:122] "RemoveContainer" containerID="2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04"
Oct 30 20:56:02.930167 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.930147    4852 scope.go:122] "RemoveContainer" containerID="2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04"
Oct 30 20:56:02.930496 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:02.930462    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04\": container with ID starting with 2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04 not found: ID does not exist" containerID="2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04"
Oct 30 20:56:02.930605 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.930501    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04"} err="failed to get container status \"2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04\": rpc error: code = NotFound desc = could not find container \"2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04\": container with ID starting with 2ce0577c156240d04891ea16a85fd9edf847cc1ceb9f0632776532d7211e1d04 not found: ID does not exist"
Oct 30 20:56:02.943227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.943204    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"]
Oct 30 20:56:02.947877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:02.947852    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln"]
Oct 30 20:56:03.441872 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.441806    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:03.548764 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.548735    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:03.587324 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.587290    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"]
Oct 30 20:56:03.587978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.587959    4852 cpu_manager.go:414] "containerMap: removing container" podUID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" containerName="main"
Oct 30 20:56:03.588030 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.587980    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" containerName="main"
Oct 30 20:56:03.588100 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.588091    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" containerName="main"
Oct 30 20:56:03.592985 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.592964    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:03.610258 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.610224    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"]
Oct 30 20:56:03.652322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.650511    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-a52bd2a1/e2e-lifecycle-model-a52bd2a1-kserve-5fcc6b5597-h92ln" podUID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.100:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 20:56:03.697546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.683735    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c5bf9a\""
Oct 30 20:56:03.698565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.698473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:03.698973 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.698915    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-c5bf9a-dockercfg-vk5n8\""
Oct 30 20:56:03.773471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.769382    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cztq7\" (UniqueName: \"kubernetes.io/projected/563df37b-be48-4126-88b0-3e3e276994f9-kube-api-access-cztq7\") pod \"payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:03.773471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.769532    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/563df37b-be48-4126-88b0-3e3e276994f9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:03.870615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.870579    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cztq7\" (UniqueName: \"kubernetes.io/projected/563df37b-be48-4126-88b0-3e3e276994f9-kube-api-access-cztq7\") pod \"payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:03.870801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.870677    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/563df37b-be48-4126-88b0-3e3e276994f9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:03.871477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.871451    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/563df37b-be48-4126-88b0-3e3e276994f9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:03.880688 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.880659    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cztq7\" (UniqueName: \"kubernetes.io/projected/563df37b-be48-4126-88b0-3e3e276994f9-kube-api-access-cztq7\") pod \"payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:03.954911 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:03.954815    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:04.007014 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.006975    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f74e3ec-1cc6-4223-9783-bef1f4fea17f" path="/var/lib/kubelet/pods/6f74e3ec-1cc6-4223-9783-bef1f4fea17f/volumes"
Oct 30 20:56:04.242968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.242937    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"]
Oct 30 20:56:04.243169 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:56:04.243150    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563df37b_be48_4126_88b0_3e3e276994f9.slice/crio-e8d5c01af8927c4f92f178458c760d3646af9e0213caed135f14a788309897a6 WatchSource:0}: Error finding container e8d5c01af8927c4f92f178458c760d3646af9e0213caed135f14a788309897a6: Status 404 returned error can't find the container with id e8d5c01af8927c4f92f178458c760d3646af9e0213caed135f14a788309897a6
Oct 30 20:56:04.630019 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.629985    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:04.713080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.712987    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"]
Oct 30 20:56:04.718351 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.718330    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:04.727698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.726714    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"]
Oct 30 20:56:04.730220 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.730192    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:04.751961 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.751926    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a61855-dockercfg-t2xmn\""
Oct 30 20:56:04.869415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.864295    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a437ec-42e5-4da7-b260-4a33f79caaae-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:04.869415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.864371    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtq2t\" (UniqueName: \"kubernetes.io/projected/83a437ec-42e5-4da7-b260-4a33f79caaae-kube-api-access-gtq2t\") pod \"payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:04.882256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.882165    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a61855\""
Oct 30 20:56:04.960287 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.960242    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb" event={"ID":"563df37b-be48-4126-88b0-3e3e276994f9","Type":"ContainerStarted","Data":"3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12"}
Oct 30 20:56:04.960786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.960308    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb" event={"ID":"563df37b-be48-4126-88b0-3e3e276994f9","Type":"ContainerStarted","Data":"e8d5c01af8927c4f92f178458c760d3646af9e0213caed135f14a788309897a6"}
Oct 30 20:56:04.960786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.960348    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:04.966294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.966261    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtq2t\" (UniqueName: \"kubernetes.io/projected/83a437ec-42e5-4da7-b260-4a33f79caaae-kube-api-access-gtq2t\") pod \"payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:04.966549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.966531    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a437ec-42e5-4da7-b260-4a33f79caaae-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:04.967196 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.967177    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a437ec-42e5-4da7-b260-4a33f79caaae-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:04.998787 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:04.998721    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb" podStartSLOduration=1.998705541 podStartE2EDuration="1.998705541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:56:04.997497912 +0000 UTC m=+1073.112232103" watchObservedRunningTime="2026-10-30 20:56:04.998705541 +0000 UTC m=+1073.113439730"
Oct 30 20:56:05.001292 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:05.001266    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtq2t\" (UniqueName: \"kubernetes.io/projected/83a437ec-42e5-4da7-b260-4a33f79caaae-kube-api-access-gtq2t\") pod \"payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:05.036648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:05.036609    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:05.322682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:05.322648    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"]
Oct 30 20:56:05.327587 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:56:05.327553    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83a437ec_42e5_4da7_b260_4a33f79caaae.slice/crio-48a5497aeedf6069a596dcf334337dd3072b8fda22044f48a53f716014d10a07 WatchSource:0}: Error finding container 48a5497aeedf6069a596dcf334337dd3072b8fda22044f48a53f716014d10a07: Status 404 returned error can't find the container with id 48a5497aeedf6069a596dcf334337dd3072b8fda22044f48a53f716014d10a07
Oct 30 20:56:05.970255 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:05.970215    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b" event={"ID":"83a437ec-42e5-4da7-b260-4a33f79caaae","Type":"ContainerStarted","Data":"87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1"}
Oct 30 20:56:05.970707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:05.970278    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b" event={"ID":"83a437ec-42e5-4da7-b260-4a33f79caaae","Type":"ContainerStarted","Data":"48a5497aeedf6069a596dcf334337dd3072b8fda22044f48a53f716014d10a07"}
Oct 30 20:56:05.970707 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:05.970309    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:05.992867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:05.992810    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b" podStartSLOduration=1.992795894 podStartE2EDuration="1.992795894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:56:05.989834966 +0000 UTC m=+1074.104569156" watchObservedRunningTime="2026-10-30 20:56:05.992795894 +0000 UTC m=+1074.107530084"
Oct 30 20:56:06.028158 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:06.028129    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:07.088149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:07.088058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:08.055758 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:08.055716    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:09.164871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:09.164825    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:09.262018 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:09.261972    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:09.357518 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:09.357482    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:10.236661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:10.236626    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:11.062295 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.062262    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:11.800755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.800722    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"]
Oct 30 20:56:11.815343 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.815297    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.824603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.824565    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c5bf9a-serving-cert\""
Oct 30 20:56:11.842858 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.842828    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"]
Oct 30 20:56:11.847687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.847656    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:11.877098 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.877048    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.877098 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.877097    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.877314 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.877157    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cn8c\" (UniqueName: \"kubernetes.io/projected/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-kube-api-access-9cn8c\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.877314 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.877212    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-metrics-certs\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.941600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.941561    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:11.978779 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.978745    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.978980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.978792    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.978980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.978893    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cn8c\" (UniqueName: \"kubernetes.io/projected/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-kube-api-access-9cn8c\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.978980 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:11.978932    4852 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-serving-cert: secret "maas-api-e2e-ipp-b-c5bf9a-serving-cert" not found
Oct 30 20:56:11.978980 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.978956    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-metrics-certs\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.979190 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:11.979016    4852 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls podName:eb93b2e4-165f-4c52-b555-6d34f5b3df9c nodeName:}" failed. No retries permitted until 2026-10-30 20:56:12.478993185 +0000 UTC m=+1080.593727367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls") pod "maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5" (UID: "eb93b2e4-165f-4c52-b555-6d34f5b3df9c") : secret "maas-api-e2e-ipp-b-c5bf9a-serving-cert" not found
Oct 30 20:56:11.979703 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.979680    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.981874 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.981847    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-metrics-certs\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:11.992221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:11.992193    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cn8c\" (UniqueName: \"kubernetes.io/projected/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-kube-api-access-9cn8c\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:12.140963 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:12.140867    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:12.331985 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:12.331952    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:12.455661 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:12.455579    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:12.486772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:12.486734    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:12.526080 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:12.526040    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:12.754371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:12.754270    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:12.923442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:12.923370    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:56:13.028256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.028225    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"]
Oct 30 20:56:13.032253 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.032231    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d03c6c9-ppd54"]
Oct 30 20:56:13.043531 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.043505    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"]
Oct 30 20:56:13.043635 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:56:13.043502    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb93b2e4_165f_4c52_b555_6d34f5b3df9c.slice/crio-34e759945cf770a6105a243d795455ce8168989f7209056ce40cf280ba7aac2b WatchSource:0}: Error finding container 34e759945cf770a6105a243d795455ce8168989f7209056ce40cf280ba7aac2b: Status 404 returned error can't find the container with id 34e759945cf770a6105a243d795455ce8168989f7209056ce40cf280ba7aac2b
Oct 30 20:56:13.078328 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.078290    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"]
Oct 30 20:56:13.084043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.084012    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.094953 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.094901    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a61855-serving-cert\""
Oct 30 20:56:13.101449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.101420    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"]
Oct 30 20:56:13.125273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.125207    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:13.216205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.201617    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adba8e8d-5a5e-445f-b223-6c3f43160905-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.216205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.201713    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv588\" (UniqueName: \"kubernetes.io/projected/adba8e8d-5a5e-445f-b223-6c3f43160905-kube-api-access-dv588\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.216205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.201807    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.216205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.202030    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-metrics-certs\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.229081 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.229032    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:13.280460 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.280363    4852 scope.go:122] "RemoveContainer" containerID="6332071a2064e033a77a420baebf45e0eedb3dd84a5c194c738c3fa1de734fba"
Oct 30 20:56:13.304429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.303866    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv588\" (UniqueName: \"kubernetes.io/projected/adba8e8d-5a5e-445f-b223-6c3f43160905-kube-api-access-dv588\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.304429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.303988    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.304429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.304097    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-metrics-certs\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.304429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.304286    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adba8e8d-5a5e-445f-b223-6c3f43160905-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.305258 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.305224    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adba8e8d-5a5e-445f-b223-6c3f43160905-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.307142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.307099    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:13.310953 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.310927    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.314938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.314883    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv588\" (UniqueName: \"kubernetes.io/projected/adba8e8d-5a5e-445f-b223-6c3f43160905-kube-api-access-dv588\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.316985 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.316920    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-metrics-certs\") pod \"maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.410516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.410483    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:13.703689 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:56:13.703649    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadba8e8d_5a5e_445f_b223_6c3f43160905.slice/crio-c22d91b34771cf6e60c577de7375a6d7a24d59fb68fb92bfd55bafed6931f607 WatchSource:0}: Error finding container c22d91b34771cf6e60c577de7375a6d7a24d59fb68fb92bfd55bafed6931f607: Status 404 returned error can't find the container with id c22d91b34771cf6e60c577de7375a6d7a24d59fb68fb92bfd55bafed6931f607
Oct 30 20:56:13.703803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:13.703720    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"]
Oct 30 20:56:14.010344 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:14.010296    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85006acd-6c86-433a-a92f-2eb6770d7373" path="/var/lib/kubelet/pods/85006acd-6c86-433a-a92f-2eb6770d7373/volumes"
Oct 30 20:56:14.040862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:14.040824    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5" event={"ID":"eb93b2e4-165f-4c52-b555-6d34f5b3df9c","Type":"ContainerStarted","Data":"8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe"}
Oct 30 20:56:14.041064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:14.040923    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5" event={"ID":"eb93b2e4-165f-4c52-b555-6d34f5b3df9c","Type":"ContainerStarted","Data":"34e759945cf770a6105a243d795455ce8168989f7209056ce40cf280ba7aac2b"}
Oct 30 20:56:14.041064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:14.040958    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:14.042365 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:14.042337    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9" event={"ID":"adba8e8d-5a5e-445f-b223-6c3f43160905","Type":"ContainerStarted","Data":"c22d91b34771cf6e60c577de7375a6d7a24d59fb68fb92bfd55bafed6931f607"}
Oct 30 20:56:14.061193 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:14.061144    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5" podStartSLOduration=2.662177582 podStartE2EDuration="3.061130148s" totalImagesPullingTime="398.952566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:56:14.058618875 +0000 UTC m=+1082.173353066" watchObservedRunningTime="2026-10-30 20:56:14.061130148 +0000 UTC m=+1082.175864394"
Oct 30 20:56:14.204259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:14.204221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:15.053702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:15.053662    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9" event={"ID":"adba8e8d-5a5e-445f-b223-6c3f43160905","Type":"ContainerStarted","Data":"e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56"}
Oct 30 20:56:15.054180 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:15.053767    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:15.099053 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:15.098992    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9" podStartSLOduration=1.701089699 podStartE2EDuration="2.098973201s" totalImagesPullingTime="397.883502ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:56:15.098162795 +0000 UTC m=+1083.212896998" watchObservedRunningTime="2026-10-30 20:56:15.098973201 +0000 UTC m=+1083.213707393"
Oct 30 20:56:15.165132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:15.165096    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:15.973263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:15.973237    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:16.121282 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:16.121248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:16.640545 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:16.640511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:16.737842 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:16.737805    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:16.980748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:16.980674    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:17.141582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:17.141540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:17.856951 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:17.856909    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:17.948066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:17.948034    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:18.539028 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:18.538997    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:19.294650 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:19.294599    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:20.059648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:20.059620    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:56:20.158421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:20.158379    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:21.014762 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:21.014729    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:21.075214 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:21.075180    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:56:21.559196 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:21.559159    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:56:21.894850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:21.894735    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:23.057173 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:23.057141    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:23.837190 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:23.837156    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:23.938617 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:23.938574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:24.430093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:24.430058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:24.532260 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:24.532202    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:24.629959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:24.629922    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:25.030514 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:25.030465    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"]
Oct 30 20:56:25.035056 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:25.034479    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d73f3b85-t2hh6"]
Oct 30 20:56:25.298838 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:25.298750    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:25.453446 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:25.453365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:25.710538 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:25.710443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:25.825119 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:25.825086    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:26.007580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:26.007552    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92fdd598-afde-49a4-a47c-2e4ae99fc977" path="/var/lib/kubelet/pods/92fdd598-afde-49a4-a47c-2e4ae99fc977/volumes"
Oct 30 20:56:27.385566 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:27.385532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:27.530443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:27.530370    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:27.667464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:27.667370    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"]
Oct 30 20:56:27.809628 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:27.809592    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:27.911231 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:27.911200    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:29.124297 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:29.124259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:30.225870 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.224525    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"]
Oct 30 20:56:30.244702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.244671    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"]
Oct 30 20:56:30.245099 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.245000    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.250877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.250852    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-351fd1-serving-cert\""
Oct 30 20:56:30.271591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.271553    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:30.355493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.355460    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:30.389912 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.389869    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-metrics-certs\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.390179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.390049    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58c8df7-9da6-468d-aa6c-41531a2f31dc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.390179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.390081    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkd8m\" (UniqueName: \"kubernetes.io/projected/f58c8df7-9da6-468d-aa6c-41531a2f31dc-kube-api-access-wkd8m\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.390353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.390327    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.469690 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.469654    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:30.492126 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.492033    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58c8df7-9da6-468d-aa6c-41531a2f31dc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.492126 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.492082    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkd8m\" (UniqueName: \"kubernetes.io/projected/f58c8df7-9da6-468d-aa6c-41531a2f31dc-kube-api-access-wkd8m\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.492589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.492372    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.492589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.492450    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-metrics-certs\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.493023 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.492947    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58c8df7-9da6-468d-aa6c-41531a2f31dc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.496608 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.496568    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.498124 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.498098    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-metrics-certs\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.505862 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.505842    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkd8m\" (UniqueName: \"kubernetes.io/projected/f58c8df7-9da6-468d-aa6c-41531a2f31dc-kube-api-access-wkd8m\") pod \"maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.572609 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.572577    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:30.605508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.605470    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:30.868293 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:56:30.868263    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58c8df7_9da6_468d_aa6c_41531a2f31dc.slice/crio-54fb8673f0dce0866ad09a591eb148476728836bac9503409d4062395aba1cbf WatchSource:0}: Error finding container 54fb8673f0dce0866ad09a591eb148476728836bac9503409d4062395aba1cbf: Status 404 returned error can't find the container with id 54fb8673f0dce0866ad09a591eb148476728836bac9503409d4062395aba1cbf
Oct 30 20:56:30.868447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:30.868373    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"]
Oct 30 20:56:31.253042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:31.252953    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr" event={"ID":"f58c8df7-9da6-468d-aa6c-41531a2f31dc","Type":"ContainerStarted","Data":"54fb8673f0dce0866ad09a591eb148476728836bac9503409d4062395aba1cbf"}
Oct 30 20:56:31.766971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:31.766919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:32.268594 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.268556    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr" event={"ID":"f58c8df7-9da6-468d-aa6c-41531a2f31dc","Type":"ContainerStarted","Data":"bc78dc34099a8abc2b81723822accba20eccf5621077daaed8c9df5c98d88acd"}
Oct 30 20:56:32.269131 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.268622    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:32.315904 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.315843    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr" podStartSLOduration=1.775919692 podStartE2EDuration="2.315824048s" totalImagesPullingTime="539.904356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:56:32.312212176 +0000 UTC m=+1100.426946378" watchObservedRunningTime="2026-10-30 20:56:32.315824048 +0000 UTC m=+1100.430558238"
Oct 30 20:56:32.353157 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.353125    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:32.527878 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.527651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:32.580409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.575027    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"]
Oct 30 20:56:32.682412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.674225    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb" podUID="563df37b-be48-4126-88b0-3e3e276994f9" containerName="payload-processing" containerID="cri-o://3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12" gracePeriod=25
Oct 30 20:56:32.735524 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.735484    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:32.826404 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:32.814444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:33.141968 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.141938    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:33.162801 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.162770    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:33.301418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.299322    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/563df37b-be48-4126-88b0-3e3e276994f9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/563df37b-be48-4126-88b0-3e3e276994f9-plugins-config-volume\") pod \"563df37b-be48-4126-88b0-3e3e276994f9\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") "
Oct 30 20:56:33.301418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.299402    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/563df37b-be48-4126-88b0-3e3e276994f9-kube-api-access-cztq7\" (UniqueName: \"kubernetes.io/projected/563df37b-be48-4126-88b0-3e3e276994f9-kube-api-access-cztq7\") pod \"563df37b-be48-4126-88b0-3e3e276994f9\" (UID: \"563df37b-be48-4126-88b0-3e3e276994f9\") "
Oct 30 20:56:33.301418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.299864    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/563df37b-be48-4126-88b0-3e3e276994f9-plugins-config-volume" pod "563df37b-be48-4126-88b0-3e3e276994f9" (UID: "563df37b-be48-4126-88b0-3e3e276994f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:56:33.301418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.300106    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/563df37b-be48-4126-88b0-3e3e276994f9-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:33.307442 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.306530    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563df37b-be48-4126-88b0-3e3e276994f9-kube-api-access-cztq7" pod "563df37b-be48-4126-88b0-3e3e276994f9" (UID: "563df37b-be48-4126-88b0-3e3e276994f9"). InnerVolumeSpecName "kube-api-access-cztq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:56:33.316624 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.316589    4852 generic.go:356] "Generic (PLEG): container finished" podID="563df37b-be48-4126-88b0-3e3e276994f9" containerID="3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12" exitCode=0
Oct 30 20:56:33.317814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.317789    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"
Oct 30 20:56:33.320399 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.317989    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb" event={"ID":"563df37b-be48-4126-88b0-3e3e276994f9","Type":"ContainerDied","Data":"3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12"}
Oct 30 20:56:33.321176 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.320470    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb" event={"ID":"563df37b-be48-4126-88b0-3e3e276994f9","Type":"ContainerDied","Data":"e8d5c01af8927c4f92f178458c760d3646af9e0213caed135f14a788309897a6"}
Oct 30 20:56:33.321176 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.320508    4852 scope.go:122] "RemoveContainer" containerID="3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12"
Oct 30 20:56:33.324287 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.324257    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:33.346615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.345831    4852 scope.go:122] "RemoveContainer" containerID="3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12"
Oct 30 20:56:33.346615 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:33.346381    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12\": container with ID starting with 3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12 not found: ID does not exist" containerID="3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12"
Oct 30 20:56:33.346615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.346453    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12"} err="failed to get container status \"3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12\": rpc error: code = NotFound desc = could not find container \"3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12\": container with ID starting with 3c969da7cbe8b81708e62207e42c7e56562d50470155b47020429044f2ba1a12 not found: ID does not exist"
Oct 30 20:56:33.361494 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.361368    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"]
Oct 30 20:56:33.367533 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.367503    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c5bf9a-7d4f95cdf5-prgwb"]
Oct 30 20:56:33.401011 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.400973    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cztq7\" (UniqueName: \"kubernetes.io/projected/563df37b-be48-4126-88b0-3e3e276994f9-kube-api-access-cztq7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:33.416590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.416553    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:33.682072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.681972    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"]
Oct 30 20:56:33.682253 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.682220    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy" containerID="cri-o://81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f" gracePeriod=30
Oct 30 20:56:33.722535 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:33.722505    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:34.016105 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:34.016067    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="563df37b-be48-4126-88b0-3e3e276994f9" path="/var/lib/kubelet/pods/563df37b-be48-4126-88b0-3e3e276994f9/volumes"
Oct 30 20:56:34.190809 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:34.190780    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:34.288772 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:34.288691    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:34.770164 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:34.770124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:35.533378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:35.533347    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:36.631774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:36.631741    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:37.563360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.563327    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:37.575399 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.575337    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b" podUID="83a437ec-42e5-4da7-b260-4a33f79caaae" containerName="payload-processing" containerID="cri-o://87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1" gracePeriod=25
Oct 30 20:56:37.873742 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.873713    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:37.984479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.984447    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83a437ec-42e5-4da7-b260-4a33f79caaae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a437ec-42e5-4da7-b260-4a33f79caaae-plugins-config-volume\") pod \"83a437ec-42e5-4da7-b260-4a33f79caaae\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") "
Oct 30 20:56:37.984655 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.984556    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83a437ec-42e5-4da7-b260-4a33f79caaae-kube-api-access-gtq2t\" (UniqueName: \"kubernetes.io/projected/83a437ec-42e5-4da7-b260-4a33f79caaae-kube-api-access-gtq2t\") pod \"83a437ec-42e5-4da7-b260-4a33f79caaae\" (UID: \"83a437ec-42e5-4da7-b260-4a33f79caaae\") "
Oct 30 20:56:37.984817 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.984793    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a437ec-42e5-4da7-b260-4a33f79caaae-plugins-config-volume" pod "83a437ec-42e5-4da7-b260-4a33f79caaae" (UID: "83a437ec-42e5-4da7-b260-4a33f79caaae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:56:37.985051 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.985032    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a437ec-42e5-4da7-b260-4a33f79caaae-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:37.986662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:37.986640    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a437ec-42e5-4da7-b260-4a33f79caaae-kube-api-access-gtq2t" pod "83a437ec-42e5-4da7-b260-4a33f79caaae" (UID: "83a437ec-42e5-4da7-b260-4a33f79caaae"). InnerVolumeSpecName "kube-api-access-gtq2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:56:38.086464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.086432    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtq2t\" (UniqueName: \"kubernetes.io/projected/83a437ec-42e5-4da7-b260-4a33f79caaae-kube-api-access-gtq2t\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:38.247258 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.247183    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:38.325176 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.325102    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:56:38.371867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.370101    4852 generic.go:356] "Generic (PLEG): container finished" podID="83a437ec-42e5-4da7-b260-4a33f79caaae" containerID="87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1" exitCode=0
Oct 30 20:56:38.371867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.370431    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b" event={"ID":"83a437ec-42e5-4da7-b260-4a33f79caaae","Type":"ContainerDied","Data":"87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1"}
Oct 30 20:56:38.371867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.370492    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b" event={"ID":"83a437ec-42e5-4da7-b260-4a33f79caaae","Type":"ContainerDied","Data":"48a5497aeedf6069a596dcf334337dd3072b8fda22044f48a53f716014d10a07"}
Oct 30 20:56:38.371867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.370523    4852 scope.go:122] "RemoveContainer" containerID="87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1"
Oct 30 20:56:38.371867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.370722    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"
Oct 30 20:56:38.386421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.386376    4852 scope.go:122] "RemoveContainer" containerID="87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1"
Oct 30 20:56:38.386839 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:38.386802    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1\": container with ID starting with 87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1 not found: ID does not exist" containerID="87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1"
Oct 30 20:56:38.386967 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.386847    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1"} err="failed to get container status \"87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1\": rpc error: code = NotFound desc = could not find container \"87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1\": container with ID starting with 87abbbd35b978f38fb984049fa2ecd98d46bf73eaaec132ae25d59c59b7e88e1 not found: ID does not exist"
Oct 30 20:56:38.398616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.398577    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"]
Oct 30 20:56:38.402872 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.402844    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a61855-7b7d46dff9-55t6b"]
Oct 30 20:56:38.960966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.960940    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:38.970711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:38.970689    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:56:39.101149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101115    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-podinfo\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.101368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101172    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-socket\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.101368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101223    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-data\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.101368 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101362    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-credential-socket\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101492    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-certs\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101530    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-token\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-token\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101649    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-kube-api-access-mk54z\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-kube-api-access-mk54z\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101707    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-envoy\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.101867    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istiod-ca-cert\") pod \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\" (UID: \"061e7d86-7f1c-40db-8ecd-76ce35bcadee\") "
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.102257    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-certs" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.102623    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istiod-ca-cert" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.102818    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.102840    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.103201    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-socket" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.103611    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-data" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:39.105710 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.103682    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-credential-socket" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:39.106364 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.105910    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-envoy" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:56:39.106364 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.105943    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-podinfo" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:56:39.106948 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.106922    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-kube-api-access-mk54z" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "kube-api-access-mk54z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:56:39.107638 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.107615    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-token" pod "061e7d86-7f1c-40db-8ecd-76ce35bcadee" (UID: "061e7d86-7f1c-40db-8ecd-76ce35bcadee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:56:39.204222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.204130    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.204222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.204162    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.204222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.204176    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk54z\" (UniqueName: \"kubernetes.io/projected/061e7d86-7f1c-40db-8ecd-76ce35bcadee-kube-api-access-mk54z\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.204222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.204188    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.204222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.204200    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.204222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.204214    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.204222 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.204227    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061e7d86-7f1c-40db-8ecd-76ce35bcadee-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:56:39.381960 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.381915    4852 generic.go:356] "Generic (PLEG): container finished" podID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerID="81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f" exitCode=0
Oct 30 20:56:39.382127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.381980    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" event={"ID":"061e7d86-7f1c-40db-8ecd-76ce35bcadee","Type":"ContainerDied","Data":"81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f"}
Oct 30 20:56:39.382127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.382008    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"
Oct 30 20:56:39.382127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.382040    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28" event={"ID":"061e7d86-7f1c-40db-8ecd-76ce35bcadee","Type":"ContainerDied","Data":"92cd3c8881d1dda98a40319bc7dfb20811ed4ee73eeb5dff080163f286838e5b"}
Oct 30 20:56:39.382127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.382070    4852 scope.go:122] "RemoveContainer" containerID="81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f"
Oct 30 20:56:39.397205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.397185    4852 scope.go:122] "RemoveContainer" containerID="30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4"
Oct 30 20:56:39.413757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.413115    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"]
Oct 30 20:56:39.413757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.413419    4852 scope.go:122] "RemoveContainer" containerID="81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f"
Oct 30 20:56:39.414005 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:39.413772    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f\": container with ID starting with 81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f not found: ID does not exist" containerID="81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f"
Oct 30 20:56:39.414005 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.413886    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f"} err="failed to get container status \"81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f\": rpc error: code = NotFound desc = could not find container \"81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f\": container with ID starting with 81406374baa7859cab6910d4c01b7477d4ba674a42ab2c5b57f47ba9a416804f not found: ID does not exist"
Oct 30 20:56:39.414005 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.413913    4852 scope.go:122] "RemoveContainer" containerID="30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4"
Oct 30 20:56:39.414255 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:39.414234    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4\": container with ID starting with 30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4 not found: ID does not exist" containerID="30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4"
Oct 30 20:56:39.414325 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.414265    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4"} err="failed to get container status \"30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4\": rpc error: code = NotFound desc = could not find container \"30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4\": container with ID starting with 30e6fe1b04789ad7416845b9de2733858c45647f4ab05d6569f20b2ff23a8db4 not found: ID does not exist"
Oct 30 20:56:39.419450 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.419429    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a52bd2a1-openshift-default-58f58875f7-hqk28"]
Oct 30 20:56:39.691043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:39.691010    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:40.005223 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:40.005148    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" path="/var/lib/kubelet/pods/061e7d86-7f1c-40db-8ecd-76ce35bcadee/volumes"
Oct 30 20:56:40.005606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:40.005591    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83a437ec-42e5-4da7-b260-4a33f79caaae" path="/var/lib/kubelet/pods/83a437ec-42e5-4da7-b260-4a33f79caaae/volumes"
Oct 30 20:56:40.364619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:40.364586    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:40.464529 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:40.464496    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:41.151155 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:41.151122    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:41.852073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:41.852037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:41.936954 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:41.936918    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:42.716590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:42.716502    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:43.758500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:43.758463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:44.693141 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.693107    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"]
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694087    4852 cpu_manager.go:414] "containerMap: removing container" podUID="563df37b-be48-4126-88b0-3e3e276994f9" containerName="payload-processing"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694113    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="563df37b-be48-4126-88b0-3e3e276994f9" containerName="payload-processing"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694137    4852 cpu_manager.go:414] "containerMap: removing container" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694144    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694167    4852 cpu_manager.go:414] "containerMap: removing container" podUID="83a437ec-42e5-4da7-b260-4a33f79caaae" containerName="payload-processing"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694175    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83a437ec-42e5-4da7-b260-4a33f79caaae" containerName="payload-processing"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694333    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694350    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="563df37b-be48-4126-88b0-3e3e276994f9" containerName="payload-processing"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694359    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy"
Oct 30 20:56:44.714436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.694370    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="83a437ec-42e5-4da7-b260-4a33f79caaae" containerName="payload-processing"
Oct 30 20:56:44.750100 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.750060    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:44.765155 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.757082    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"]
Oct 30 20:56:44.765155 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.757190    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:44.765155 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.762250    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5452c3-serving-cert\""
Oct 30 20:56:44.844421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.841759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:44.945471 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.944990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:44.961437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.960049    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a3e7c1-f477-4c63-9b4d-da465369f449-openshift-service-ca\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:44.961437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.960323    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqcw7\" (UniqueName: \"kubernetes.io/projected/95a3e7c1-f477-4c63-9b4d-da465369f449-kube-api-access-dqcw7\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:44.961437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.960424    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-metrics-certs\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:44.961437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:44.960483    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-maas-api-tls\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.062900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.062858    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqcw7\" (UniqueName: \"kubernetes.io/projected/95a3e7c1-f477-4c63-9b4d-da465369f449-kube-api-access-dqcw7\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.063300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.063264    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-metrics-certs\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.063469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.063363    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-maas-api-tls\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.063469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.063463    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a3e7c1-f477-4c63-9b4d-da465369f449-openshift-service-ca\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.064649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.064619    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a3e7c1-f477-4c63-9b4d-da465369f449-openshift-service-ca\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.066144 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.066115    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-metrics-certs\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.066821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.066799    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-maas-api-tls\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.100416 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.097257    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqcw7\" (UniqueName: \"kubernetes.io/projected/95a3e7c1-f477-4c63-9b4d-da465369f449-kube-api-access-dqcw7\") pod \"maas-api-e2e-api-b-5452c3-656b7d7496-c2txk\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.144526 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.144492    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:45.395746 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.394327    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:45.401877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.401278    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:45.787578 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:45.787546    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"]
Oct 30 20:56:45.789168 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:56:45.789144    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a3e7c1_f477_4c63_9b4d_da465369f449.slice/crio-f18d98f206d10d5cd62f86cf77e4c48edef2c825c67d0bece071e24fecfcd928 WatchSource:0}: Error finding container f18d98f206d10d5cd62f86cf77e4c48edef2c825c67d0bece071e24fecfcd928: Status 404 returned error can't find the container with id f18d98f206d10d5cd62f86cf77e4c48edef2c825c67d0bece071e24fecfcd928
Oct 30 20:56:46.147537 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:46.147461    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:46.255263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:46.255234    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:46.333359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:46.333336    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:46.511246 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:46.511217    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk" event={"ID":"95a3e7c1-f477-4c63-9b4d-da465369f449","Type":"ContainerStarted","Data":"dd92e733df0ba7e818999196207dfe7bc2a88039d6fbd9c61c4a1e967f62a0d1"}
Oct 30 20:56:46.511443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:46.511268    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk" event={"ID":"95a3e7c1-f477-4c63-9b4d-da465369f449","Type":"ContainerStarted","Data":"f18d98f206d10d5cd62f86cf77e4c48edef2c825c67d0bece071e24fecfcd928"}
Oct 30 20:56:46.511497 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:46.511444    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:46.537199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:46.537136    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk" podStartSLOduration=2.122799025 podStartE2EDuration="2.537116721s" totalImagesPullingTime="414.317696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:56:46.530301596 +0000 UTC m=+1114.645035783" watchObservedRunningTime="2026-10-30 20:56:46.537116721 +0000 UTC m=+1114.651850918"
Oct 30 20:56:47.808421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:47.808368    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:49.563786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:49.563754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:50.910017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:50.909983    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:51.559734 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:56:51.559699    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:56:51.961632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:51.961553    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:52.283024 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:52.282990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:52.375908 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:52.375873    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:52.528751 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:52.528721    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:56:52.570843 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:52.570744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:53.415936 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:53.415884    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:54.651170 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:54.651135    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:55.493286 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:55.493245    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:56.502856 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:56.502822    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:57.486030 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:57.485990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:58.443074 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:58.443030    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:56:59.322887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:56:59.322849    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:00.200295 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:00.200264    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:01.222911 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:01.222868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:02.860257 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:02.860225    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:03.641231 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:03.641190    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:03.741635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:03.741607    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:03.842598 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:03.842563    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:04.696886 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:04.696853    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:05.780867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:05.780821    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:06.120582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:06.120506    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:06.755648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:06.755614    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:07.110539 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:07.110458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:08.431533 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:08.431500    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:09.262373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:09.262338    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:09.359007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:09.358969    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:09.458559 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:09.458529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:09.660118 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:09.660030    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:09.980480 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:09.980338    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:10.079540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:10.079490    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:10.469988 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:10.469956    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:11.026616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:11.026582    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"]
Oct 30 20:57:11.026616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:11.026595    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:11.030407 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:11.030371    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66537acb-slk2p"]
Oct 30 20:57:11.824540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:11.824505    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:12.006051 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:12.006017    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2305a0ae-5164-4316-9d9e-c5419e2c1135" path="/var/lib/kubelet/pods/2305a0ae-5164-4316-9d9e-c5419e2c1135/volumes"
Oct 30 20:57:12.559337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:12.559305    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:13.190508 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:13.190477    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:13.322612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:13.322577    4852 scope.go:122] "RemoveContainer" containerID="ad3a026679ad077e4ee9250e221e065e87334bf28fad9b23518f43e08fb38303"
Oct 30 20:57:13.333510 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:13.333484    4852 scope.go:122] "RemoveContainer" containerID="c6c9606d1e5a348cebe7f079eb730c9ec8324cfc74b2bf355780e60e893dbe15"
Oct 30 20:57:13.449089 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:13.449003    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:14.429343 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:14.429309    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:15.225115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:15.225073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:16.165658 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:16.165625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:16.900693 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:16.900662    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:17.688242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:17.688211    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"]
Oct 30 20:57:17.752839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:17.752795    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:17.855006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:17.854973    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:17.955623 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:17.955545    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:18.223264 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:18.223137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:18.641575 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:18.641540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:19.025107 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:19.025072    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"]
Oct 30 20:57:19.034493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:19.034197    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bc4c14da-cbhn7"]
Oct 30 20:57:19.655530 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:19.655495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:20.005854 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:20.005828    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08971e8f-6005-4e56-b9e1-32deb5d3c2d6" path="/var/lib/kubelet/pods/08971e8f-6005-4e56-b9e1-32deb5d3c2d6/volumes"
Oct 30 20:57:21.321946 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:21.321912    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:21.561083 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:57:21.561047    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:57:22.319614 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:22.319232    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:22.400360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:22.400319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:22.544437 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:22.538376    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:22.689154 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:22.689050    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz" podUID="a117995e-6ced-4d79-b4be-2e7604d0a8d6" containerName="payload-processing" containerID="cri-o://5af1332e1e01e54a9feb0c4053cd79a37a987d83b25049c5f0541f2afdb8d524" gracePeriod=25
Oct 30 20:57:22.832627 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:22.832589    4852 generic.go:356] "Generic (PLEG): container finished" podID="a117995e-6ced-4d79-b4be-2e7604d0a8d6" containerID="5af1332e1e01e54a9feb0c4053cd79a37a987d83b25049c5f0541f2afdb8d524" exitCode=0
Oct 30 20:57:22.832790 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:22.832655    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz" event={"ID":"a117995e-6ced-4d79-b4be-2e7604d0a8d6","Type":"ContainerDied","Data":"5af1332e1e01e54a9feb0c4053cd79a37a987d83b25049c5f0541f2afdb8d524"}
Oct 30 20:57:22.912308 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:22.912269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:23.054591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.054559    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:57:23.222716 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.222687    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:23.249986 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.249956    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a117995e-6ced-4d79-b4be-2e7604d0a8d6-kube-api-access-k8rhn\" (UniqueName: \"kubernetes.io/projected/a117995e-6ced-4d79-b4be-2e7604d0a8d6-kube-api-access-k8rhn\") pod \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") "
Oct 30 20:57:23.250145 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.250086    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a117995e-6ced-4d79-b4be-2e7604d0a8d6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a117995e-6ced-4d79-b4be-2e7604d0a8d6-plugins-config-volume\") pod \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\" (UID: \"a117995e-6ced-4d79-b4be-2e7604d0a8d6\") "
Oct 30 20:57:23.250537 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.250506    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a117995e-6ced-4d79-b4be-2e7604d0a8d6-plugins-config-volume" pod "a117995e-6ced-4d79-b4be-2e7604d0a8d6" (UID: "a117995e-6ced-4d79-b4be-2e7604d0a8d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:57:23.252505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.252476    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a117995e-6ced-4d79-b4be-2e7604d0a8d6-kube-api-access-k8rhn" pod "a117995e-6ced-4d79-b4be-2e7604d0a8d6" (UID: "a117995e-6ced-4d79-b4be-2e7604d0a8d6"). InnerVolumeSpecName "kube-api-access-k8rhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:57:23.351946 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.351903    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a117995e-6ced-4d79-b4be-2e7604d0a8d6-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:23.351946 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.351947    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8rhn\" (UniqueName: \"kubernetes.io/projected/a117995e-6ced-4d79-b4be-2e7604d0a8d6-kube-api-access-k8rhn\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:23.506359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.506325    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:23.844780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.844755    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"
Oct 30 20:57:23.844991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.844745    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz" event={"ID":"a117995e-6ced-4d79-b4be-2e7604d0a8d6","Type":"ContainerDied","Data":"134b0200f4204ea8cc842fb62e5a060868d1d0ba56852eda0d949a797f7a715a"}
Oct 30 20:57:23.844991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.844939    4852 scope.go:122] "RemoveContainer" containerID="5af1332e1e01e54a9feb0c4053cd79a37a987d83b25049c5f0541f2afdb8d524"
Oct 30 20:57:23.872073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.872040    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"]
Oct 30 20:57:23.875870 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.875847    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz"]
Oct 30 20:57:23.921839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:23.921804    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-319188-59f459b575-r26vz" podUID="a117995e-6ced-4d79-b4be-2e7604d0a8d6" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.108:50052\" within 1s: context deadline exceeded"
Oct 30 20:57:23.921963 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:57:23.921875    4852 logging.go:55] [core] [Channel #1421 SubChannel #1422]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.108:50052", ServerName: "10.133.0.108:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.108:50052: operation was canceled"
Oct 30 20:57:24.005073 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:24.005039    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a117995e-6ced-4d79-b4be-2e7604d0a8d6" path="/var/lib/kubelet/pods/a117995e-6ced-4d79-b4be-2e7604d0a8d6/volumes"
Oct 30 20:57:24.293777 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:24.293742    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:25.141616 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:25.141578    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:26.171056 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:26.171023    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:27.022657 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:27.022562    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:27.743478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:27.743413    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:28.571660 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:28.571631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:29.309298 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:29.309250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:30.006434 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:30.006366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:30.103965 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:30.103924    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:30.330719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:30.330627    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:30.412373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:30.412339    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:31.043267 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.043229    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"]
Oct 30 20:57:31.044003 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.043968    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d" podUID="4232a13e-0d80-4c2f-9497-0d5c944369a1" containerName="maas-api" containerID="cri-o://552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe" gracePeriod=30
Oct 30 20:57:31.104602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.099318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:31.204028 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.192759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:31.292774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.292738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:31.409458 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.403115    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:57:31.487750 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.487714    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4232a13e-0d80-4c2f-9497-0d5c944369a1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4232a13e-0d80-4c2f-9497-0d5c944369a1-openshift-service-ca\") pod \"4232a13e-0d80-4c2f-9497-0d5c944369a1\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") "
Oct 30 20:57:31.488003 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.487765    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-metrics-certs\") pod \"4232a13e-0d80-4c2f-9497-0d5c944369a1\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") "
Oct 30 20:57:31.488003 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.487786    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-maas-api-tls\") pod \"4232a13e-0d80-4c2f-9497-0d5c944369a1\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") "
Oct 30 20:57:31.488003 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.487945    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4232a13e-0d80-4c2f-9497-0d5c944369a1-kube-api-access-rctf7\" (UniqueName: \"kubernetes.io/projected/4232a13e-0d80-4c2f-9497-0d5c944369a1-kube-api-access-rctf7\") pod \"4232a13e-0d80-4c2f-9497-0d5c944369a1\" (UID: \"4232a13e-0d80-4c2f-9497-0d5c944369a1\") "
Oct 30 20:57:31.489333 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.489302    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4232a13e-0d80-4c2f-9497-0d5c944369a1-openshift-service-ca" pod "4232a13e-0d80-4c2f-9497-0d5c944369a1" (UID: "4232a13e-0d80-4c2f-9497-0d5c944369a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:57:31.495741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.495713    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-metrics-certs" pod "4232a13e-0d80-4c2f-9497-0d5c944369a1" (UID: "4232a13e-0d80-4c2f-9497-0d5c944369a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:31.496118 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.496088    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4232a13e-0d80-4c2f-9497-0d5c944369a1-kube-api-access-rctf7" pod "4232a13e-0d80-4c2f-9497-0d5c944369a1" (UID: "4232a13e-0d80-4c2f-9497-0d5c944369a1"). InnerVolumeSpecName "kube-api-access-rctf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:57:31.497135 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.497106    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-maas-api-tls" pod "4232a13e-0d80-4c2f-9497-0d5c944369a1" (UID: "4232a13e-0d80-4c2f-9497-0d5c944369a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:31.538077 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.538037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:31.589755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.589673    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4232a13e-0d80-4c2f-9497-0d5c944369a1-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:31.589755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.589703    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:31.589755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.589713    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4232a13e-0d80-4c2f-9497-0d5c944369a1-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:31.589755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.589722    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rctf7\" (UniqueName: \"kubernetes.io/projected/4232a13e-0d80-4c2f-9497-0d5c944369a1-kube-api-access-rctf7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:31.810677 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.810645    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:31.918596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.918489    4852 generic.go:356] "Generic (PLEG): container finished" podID="4232a13e-0d80-4c2f-9497-0d5c944369a1" containerID="552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe" exitCode=0
Oct 30 20:57:31.918596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.918543    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"
Oct 30 20:57:31.918596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.918573    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d" event={"ID":"4232a13e-0d80-4c2f-9497-0d5c944369a1","Type":"ContainerDied","Data":"552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe"}
Oct 30 20:57:31.918899 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.918640    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d" event={"ID":"4232a13e-0d80-4c2f-9497-0d5c944369a1","Type":"ContainerDied","Data":"c0cb5ec463f928af5c07797d5dc9d639c4eb2401fae148858258f65bccf28b4a"}
Oct 30 20:57:31.918899 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.918667    4852 scope.go:122] "RemoveContainer" containerID="552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe"
Oct 30 20:57:31.937759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.937731    4852 scope.go:122] "RemoveContainer" containerID="552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe"
Oct 30 20:57:31.938177 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:57:31.938145    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe\": container with ID starting with 552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe not found: ID does not exist" containerID="552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe"
Oct 30 20:57:31.938315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.938188    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe"} err="failed to get container status \"552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe\": rpc error: code = NotFound desc = could not find container \"552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe\": container with ID starting with 552f2e99e4629506aeb787cabd8a13a265bd7b26f4c708a84e84441244e05dbe not found: ID does not exist"
Oct 30 20:57:31.947817 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.947754    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"]
Oct 30 20:57:31.956054 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.956029    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7c14e2-5cfd868798-28f4d"]
Oct 30 20:57:31.989714 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:31.989674    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:32.006563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:32.006533    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4232a13e-0d80-4c2f-9497-0d5c944369a1" path="/var/lib/kubelet/pods/4232a13e-0d80-4c2f-9497-0d5c944369a1/volumes"
Oct 30 20:57:32.349515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:32.349479    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:32.828152 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:32.828112    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:33.017312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:33.017275    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:33.273470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:33.273430    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:33.588929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:33.588832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:33.903642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:33.903508    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:34.826042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:34.826010    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:35.480766 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.480417    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"]
Oct 30 20:57:35.485712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.485658    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5" podUID="eb93b2e4-165f-4c52-b555-6d34f5b3df9c" containerName="maas-api" containerID="cri-o://8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe" gracePeriod=30
Oct 30 20:57:35.519859 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.519803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:35.620502 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.620471    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:35.724122 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.724090    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:35.781427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.781358    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:57:35.827871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.827799    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:35.862043 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.861983    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-metrics-certs\") pod \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") "
Oct 30 20:57:35.862263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.862184    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-kube-api-access-9cn8c\" (UniqueName: \"kubernetes.io/projected/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-kube-api-access-9cn8c\") pod \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") "
Oct 30 20:57:35.862340 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.862299    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-openshift-service-ca\") pod \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") "
Oct 30 20:57:35.862489 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.862471    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls\") pod \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\" (UID: \"eb93b2e4-165f-4c52-b555-6d34f5b3df9c\") "
Oct 30 20:57:35.864198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.864143    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-openshift-service-ca" pod "eb93b2e4-165f-4c52-b555-6d34f5b3df9c" (UID: "eb93b2e4-165f-4c52-b555-6d34f5b3df9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:57:35.875256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.875203    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-metrics-certs" pod "eb93b2e4-165f-4c52-b555-6d34f5b3df9c" (UID: "eb93b2e4-165f-4c52-b555-6d34f5b3df9c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:35.875606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.875572    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls" pod "eb93b2e4-165f-4c52-b555-6d34f5b3df9c" (UID: "eb93b2e4-165f-4c52-b555-6d34f5b3df9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:35.878202 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.878163    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-kube-api-access-9cn8c" pod "eb93b2e4-165f-4c52-b555-6d34f5b3df9c" (UID: "eb93b2e4-165f-4c52-b555-6d34f5b3df9c"). InnerVolumeSpecName "kube-api-access-9cn8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:57:35.910643 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.910600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:35.952752 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.952716    4852 generic.go:356] "Generic (PLEG): container finished" podID="eb93b2e4-165f-4c52-b555-6d34f5b3df9c" containerID="8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe" exitCode=0
Oct 30 20:57:35.952971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.952784    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"
Oct 30 20:57:35.952971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.952803    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5" event={"ID":"eb93b2e4-165f-4c52-b555-6d34f5b3df9c","Type":"ContainerDied","Data":"8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe"}
Oct 30 20:57:35.952971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.952876    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5" event={"ID":"eb93b2e4-165f-4c52-b555-6d34f5b3df9c","Type":"ContainerDied","Data":"34e759945cf770a6105a243d795455ce8168989f7209056ce40cf280ba7aac2b"}
Oct 30 20:57:35.952971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.952907    4852 scope.go:122] "RemoveContainer" containerID="8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe"
Oct 30 20:57:35.965877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.965733    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:35.965877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.965762    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:35.965877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.965779    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:35.965877 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.965788    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cn8c\" (UniqueName: \"kubernetes.io/projected/eb93b2e4-165f-4c52-b555-6d34f5b3df9c-kube-api-access-9cn8c\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:35.968470 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.968448    4852 scope.go:122] "RemoveContainer" containerID="8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe"
Oct 30 20:57:35.968858 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:57:35.968839    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe\": container with ID starting with 8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe not found: ID does not exist" containerID="8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe"
Oct 30 20:57:35.968932 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.968870    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe"} err="failed to get container status \"8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe\": rpc error: code = NotFound desc = could not find container \"8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe\": container with ID starting with 8043ab6f1d32b6060c81eaa0ba0576f699655263ad1e617c81beeb13233a9cbe not found: ID does not exist"
Oct 30 20:57:35.982667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.982631    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"]
Oct 30 20:57:35.993424 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:35.990042    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c5bf9a-5748fb75b7-xkvh5"]
Oct 30 20:57:36.020285 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:36.020243    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb93b2e4-165f-4c52-b555-6d34f5b3df9c" path="/var/lib/kubelet/pods/eb93b2e4-165f-4c52-b555-6d34f5b3df9c/volumes"
Oct 30 20:57:36.396322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:36.396285    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:36.497127 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:36.497077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:37.512362 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:37.512311    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:38.323104 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:38.323060    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:39.340225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:39.340194    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:39.418287 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:39.417298    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:39.672523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:39.672444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:39.875017 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:39.874976    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:40.959378 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:40.959341    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:41.030475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:41.030438    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"]
Oct 30 20:57:41.034035 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:41.034011    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-63ba1000-rshkj"]
Oct 30 20:57:41.060646 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:41.060612    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:41.348926 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:41.348895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:42.006836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:42.006799    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="817e8b83-d053-49d3-901b-d0220b10b9ac" path="/var/lib/kubelet/pods/817e8b83-d053-49d3-901b-d0220b10b9ac/volumes"
Oct 30 20:57:42.426331 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:42.426248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:43.170706 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:43.170658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:43.271456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:43.271422    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:43.537311 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:43.537276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:44.285263 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:44.285228    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:45.120858 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:45.120825    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:45.216008 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:45.215972    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:45.526901 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:45.526863    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:45.703984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:45.703949    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:46.002949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.002859    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"]
Oct 30 20:57:46.003180 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.003152    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9" podUID="adba8e8d-5a5e-445f-b223-6c3f43160905" containerName="maas-api" containerID="cri-o://e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56" gracePeriod=30
Oct 30 20:57:46.061949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.061911    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9" podUID="adba8e8d-5a5e-445f-b223-6c3f43160905" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.112:8443/health\": dial tcp 10.133.0.112:8443: connect: connection refused"
Oct 30 20:57:46.080304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.080275    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:46.177065 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.177029    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:46.267081 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.267049    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:46.299000 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.298972    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:57:46.419282 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.419240    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:46.465662 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.465628    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-maas-api-tls\") pod \"adba8e8d-5a5e-445f-b223-6c3f43160905\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") "
Oct 30 20:57:46.465851 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.465680    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-metrics-certs\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-metrics-certs\") pod \"adba8e8d-5a5e-445f-b223-6c3f43160905\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") "
Oct 30 20:57:46.465957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.465923    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/adba8e8d-5a5e-445f-b223-6c3f43160905-kube-api-access-dv588\" (UniqueName: \"kubernetes.io/projected/adba8e8d-5a5e-445f-b223-6c3f43160905-kube-api-access-dv588\") pod \"adba8e8d-5a5e-445f-b223-6c3f43160905\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") "
Oct 30 20:57:46.466179 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.466158    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/adba8e8d-5a5e-445f-b223-6c3f43160905-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adba8e8d-5a5e-445f-b223-6c3f43160905-openshift-service-ca\") pod \"adba8e8d-5a5e-445f-b223-6c3f43160905\" (UID: \"adba8e8d-5a5e-445f-b223-6c3f43160905\") "
Oct 30 20:57:46.466597 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.466570    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adba8e8d-5a5e-445f-b223-6c3f43160905-openshift-service-ca" pod "adba8e8d-5a5e-445f-b223-6c3f43160905" (UID: "adba8e8d-5a5e-445f-b223-6c3f43160905"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:57:46.467240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.467218    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adba8e8d-5a5e-445f-b223-6c3f43160905-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:46.469192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.469165    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-metrics-certs" pod "adba8e8d-5a5e-445f-b223-6c3f43160905" (UID: "adba8e8d-5a5e-445f-b223-6c3f43160905"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:46.469337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.469259    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-maas-api-tls" pod "adba8e8d-5a5e-445f-b223-6c3f43160905" (UID: "adba8e8d-5a5e-445f-b223-6c3f43160905"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:46.469337 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.469267    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adba8e8d-5a5e-445f-b223-6c3f43160905-kube-api-access-dv588" pod "adba8e8d-5a5e-445f-b223-6c3f43160905" (UID: "adba8e8d-5a5e-445f-b223-6c3f43160905"). InnerVolumeSpecName "kube-api-access-dv588". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:57:46.568546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.568464    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:46.568546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.568493    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adba8e8d-5a5e-445f-b223-6c3f43160905-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:46.568546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.568505    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv588\" (UniqueName: \"kubernetes.io/projected/adba8e8d-5a5e-445f-b223-6c3f43160905-kube-api-access-dv588\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:46.573669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.573642    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:46.768966 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:46.768934    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:47.052912 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.052881    4852 generic.go:356] "Generic (PLEG): container finished" podID="adba8e8d-5a5e-445f-b223-6c3f43160905" containerID="e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56" exitCode=0
Oct 30 20:57:47.053095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.052931    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"
Oct 30 20:57:47.053095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.052978    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9" event={"ID":"adba8e8d-5a5e-445f-b223-6c3f43160905","Type":"ContainerDied","Data":"e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56"}
Oct 30 20:57:47.053095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.053030    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9" event={"ID":"adba8e8d-5a5e-445f-b223-6c3f43160905","Type":"ContainerDied","Data":"c22d91b34771cf6e60c577de7375a6d7a24d59fb68fb92bfd55bafed6931f607"}
Oct 30 20:57:47.053095 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.053055    4852 scope.go:122] "RemoveContainer" containerID="e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56"
Oct 30 20:57:47.065411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.065379    4852 scope.go:122] "RemoveContainer" containerID="e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56"
Oct 30 20:57:47.065752 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:57:47.065726    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56\": container with ID starting with e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56 not found: ID does not exist" containerID="e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56"
Oct 30 20:57:47.065836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.065759    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56"} err="failed to get container status \"e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56\": rpc error: code = NotFound desc = could not find container \"e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56\": container with ID starting with e9bbb58000bc4aa733630e305d38b7bb5498acc75256bae89cd38b0696c8bc56 not found: ID does not exist"
Oct 30 20:57:47.078888 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.078861    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"]
Oct 30 20:57:47.088949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.088906    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a61855-fb94f6665-24cb9"]
Oct 30 20:57:47.212142 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.212109    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:47.770994 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:47.770960    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:48.004971 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:48.004938    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="adba8e8d-5a5e-445f-b223-6c3f43160905" path="/var/lib/kubelet/pods/adba8e8d-5a5e-445f-b223-6c3f43160905/volumes"
Oct 30 20:57:48.424102 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:48.424064    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:48.792560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:48.792524    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:49.222259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:49.222177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:49.324907 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:49.324852    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:49.801245 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:49.801187    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:50.458739 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:50.458699    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:50.586436 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:50.586402    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:51.217359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:51.217327    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:51.559164 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:57:51.559128    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:57:52.103420 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:52.103358    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:52.291865 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:52.291829    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:53.138016 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:53.137982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:53.922108 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:53.922068    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:54.122006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:54.121976    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:54.222937 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:54.222827    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:54.625626 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:54.625582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:54.863507 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:54.863455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:55.921975 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:55.921941    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:56.032477 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:56.032448    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:56.465884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:56.465850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:56.549497 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:56.549458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:57.008592 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:57.008561    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:57.109107 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:57.109072    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:57.927821 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:57.927790    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"]
Oct 30 20:57:57.928072 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:57.928009    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr" podUID="f58c8df7-9da6-468d-aa6c-41531a2f31dc" containerName="maas-api" containerID="cri-o://bc78dc34099a8abc2b81723822accba20eccf5621077daaed8c9df5c98d88acd" gracePeriod=30
Oct 30 20:57:57.973786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:57.973752    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:58.107042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.107011    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:58.179277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.179199    4852 generic.go:356] "Generic (PLEG): container finished" podID="f58c8df7-9da6-468d-aa6c-41531a2f31dc" containerID="bc78dc34099a8abc2b81723822accba20eccf5621077daaed8c9df5c98d88acd" exitCode=0
Oct 30 20:57:58.179277 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.179267    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr" event={"ID":"f58c8df7-9da6-468d-aa6c-41531a2f31dc","Type":"ContainerDied","Data":"bc78dc34099a8abc2b81723822accba20eccf5621077daaed8c9df5c98d88acd"}
Oct 30 20:57:58.197736 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.197701    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:58.240217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.240189    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:57:58.288696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.288661    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:58.368059 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.368022    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-maas-api-tls\") pod \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") "
Oct 30 20:57:58.368255 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.368214    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f58c8df7-9da6-468d-aa6c-41531a2f31dc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58c8df7-9da6-468d-aa6c-41531a2f31dc-openshift-service-ca\") pod \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") "
Oct 30 20:57:58.368330 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.368257    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-metrics-certs\") pod \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") "
Oct 30 20:57:58.368481 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.368462    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f58c8df7-9da6-468d-aa6c-41531a2f31dc-kube-api-access-wkd8m\" (UniqueName: \"kubernetes.io/projected/f58c8df7-9da6-468d-aa6c-41531a2f31dc-kube-api-access-wkd8m\") pod \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\" (UID: \"f58c8df7-9da6-468d-aa6c-41531a2f31dc\") "
Oct 30 20:57:58.368633 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.368604    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f58c8df7-9da6-468d-aa6c-41531a2f31dc-openshift-service-ca" pod "f58c8df7-9da6-468d-aa6c-41531a2f31dc" (UID: "f58c8df7-9da6-468d-aa6c-41531a2f31dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:57:58.369025 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.369004    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f58c8df7-9da6-468d-aa6c-41531a2f31dc-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:58.370284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.370258    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-maas-api-tls" pod "f58c8df7-9da6-468d-aa6c-41531a2f31dc" (UID: "f58c8df7-9da6-468d-aa6c-41531a2f31dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:58.370472 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.370448    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58c8df7-9da6-468d-aa6c-41531a2f31dc-kube-api-access-wkd8m" pod "f58c8df7-9da6-468d-aa6c-41531a2f31dc" (UID: "f58c8df7-9da6-468d-aa6c-41531a2f31dc"). InnerVolumeSpecName "kube-api-access-wkd8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:57:58.370784 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.370759    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-metrics-certs" pod "f58c8df7-9da6-468d-aa6c-41531a2f31dc" (UID: "f58c8df7-9da6-468d-aa6c-41531a2f31dc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:57:58.377914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.377878    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:58.474401 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.472828    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:58.474401 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.472865    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkd8m\" (UniqueName: \"kubernetes.io/projected/f58c8df7-9da6-468d-aa6c-41531a2f31dc-kube-api-access-wkd8m\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:58.474401 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.472893    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f58c8df7-9da6-468d-aa6c-41531a2f31dc-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:57:58.498418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.497200    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:58.806426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.806366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:58.904625 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:58.904588    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:57:59.192217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.192125    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"
Oct 30 20:57:59.192217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.192134    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr" event={"ID":"f58c8df7-9da6-468d-aa6c-41531a2f31dc","Type":"ContainerDied","Data":"54fb8673f0dce0866ad09a591eb148476728836bac9503409d4062395aba1cbf"}
Oct 30 20:57:59.192217 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.192208    4852 scope.go:122] "RemoveContainer" containerID="bc78dc34099a8abc2b81723822accba20eccf5621077daaed8c9df5c98d88acd"
Oct 30 20:57:59.251679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.251645    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"]
Oct 30 20:57:59.262251 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.262218    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-351fd1-5bcf87588f-dhnlr"]
Oct 30 20:57:59.837698 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.837664    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"]
Oct 30 20:57:59.838134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.838104    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy" containerID="cri-o://bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c" gracePeriod=30
Oct 30 20:57:59.846005 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:57:59.845971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:00.009020 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:00.008989    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f58c8df7-9da6-468d-aa6c-41531a2f31dc" path="/var/lib/kubelet/pods/f58c8df7-9da6-468d-aa6c-41531a2f31dc/volumes"
Oct 30 20:58:00.671595 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:00.671556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:00.842294 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:00.842265    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:01.046398 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:01.046364    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:01.818195 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:01.818112    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:01.941984 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:01.941947    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:02.071509 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:02.071325    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:02.825399 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:02.825353    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:03.674998 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:03.674963    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:03.785922 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:03.785885    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:04.496186 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:04.496154    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:05.029769 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.029742    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"]
Oct 30 20:58:05.029996 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.029970    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk" podUID="95a3e7c1-f477-4c63-9b4d-da465369f449" containerName="maas-api" containerID="cri-o://dd92e733df0ba7e818999196207dfe7bc2a88039d6fbd9c61c4a1e967f62a0d1" gracePeriod=30
Oct 30 20:58:05.057432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.057379    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:05.172122 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.167632    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:58:05.252468 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.252437    4852 generic.go:356] "Generic (PLEG): container finished" podID="4214ba5f-63d1-4448-abad-833ebbf18022" containerID="bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c" exitCode=0
Oct 30 20:58:05.252649 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.252630    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" event={"ID":"4214ba5f-63d1-4448-abad-833ebbf18022","Type":"ContainerDied","Data":"bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c"}
Oct 30 20:58:05.252699 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.252688    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q" event={"ID":"4214ba5f-63d1-4448-abad-833ebbf18022","Type":"ContainerDied","Data":"9d19c877a3791e4b7d4a39c22ec37a8bbedf8baf94b7bb5f6212e7741593555d"}
Oct 30 20:58:05.252737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.252720    4852 scope.go:122] "RemoveContainer" containerID="bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c"
Oct 30 20:58:05.252844 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.252832    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"
Oct 30 20:58:05.255716 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.255689    4852 generic.go:356] "Generic (PLEG): container finished" podID="95a3e7c1-f477-4c63-9b4d-da465369f449" containerID="dd92e733df0ba7e818999196207dfe7bc2a88039d6fbd9c61c4a1e967f62a0d1" exitCode=0
Oct 30 20:58:05.255860 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.255785    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk" event={"ID":"95a3e7c1-f477-4c63-9b4d-da465369f449","Type":"ContainerDied","Data":"dd92e733df0ba7e818999196207dfe7bc2a88039d6fbd9c61c4a1e967f62a0d1"}
Oct 30 20:58:05.267291 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.267263    4852 scope.go:122] "RemoveContainer" containerID="bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c"
Oct 30 20:58:05.267699 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:58:05.267673    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c\": container with ID starting with bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c not found: ID does not exist" containerID="bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c"
Oct 30 20:58:05.267812 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.267712    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c"} err="failed to get container status \"bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c\": rpc error: code = NotFound desc = could not find container \"bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c\": container with ID starting with bd00a9008825163ddacfa5ad0943eeca0c157eeeabbb4b7f244d1612fff49a3c not found: ID does not exist"
Oct 30 20:58:05.283244 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283215    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-certs\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283289    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-istio-token\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-istio-token\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283451 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283379    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-kube-api-access-m6dm6\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-kube-api-access-m6dm6\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283451 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283435    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-certs" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:05.283525 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283499    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4214ba5f-63d1-4448-abad-833ebbf18022-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4214ba5f-63d1-4448-abad-833ebbf18022-istio-podinfo\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283533    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-data\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283651    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-socket\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283738    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-envoy\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283818 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283776    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4214ba5f-63d1-4448-abad-833ebbf18022-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4214ba5f-63d1-4448-abad-833ebbf18022-istiod-ca-cert\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283874 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283853    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-credential-socket\") pod \"4214ba5f-63d1-4448-abad-833ebbf18022\" (UID: \"4214ba5f-63d1-4448-abad-833ebbf18022\") "
Oct 30 20:58:05.283931 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283872    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-data" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:05.283991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.283938    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-socket" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:05.284261 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.284235    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-credential-socket" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:05.284360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.284172    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4214ba5f-63d1-4448-abad-833ebbf18022-istiod-ca-cert" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:58:05.284590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.284566    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4214ba5f-63d1-4448-abad-833ebbf18022-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.284669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.284598    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.284669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.284614    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.284669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.284630    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.284669 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.284643    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.285834 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.285812    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4214ba5f-63d1-4448-abad-833ebbf18022-istio-podinfo" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:58:05.286479 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.286458    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-istio-token" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:58:05.287341 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.287322    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-envoy" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:05.287418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.287357    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-kube-api-access-m6dm6" pod "4214ba5f-63d1-4448-abad-833ebbf18022" (UID: "4214ba5f-63d1-4448-abad-833ebbf18022"). InnerVolumeSpecName "kube-api-access-m6dm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:58:05.291593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.291545    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:05.316540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.316516    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:58:05.386184 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.386094    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-maas-api-tls\") pod \"95a3e7c1-f477-4c63-9b4d-da465369f449\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") "
Oct 30 20:58:05.386184 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.386159    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95a3e7c1-f477-4c63-9b4d-da465369f449-kube-api-access-dqcw7\" (UniqueName: \"kubernetes.io/projected/95a3e7c1-f477-4c63-9b4d-da465369f449-kube-api-access-dqcw7\") pod \"95a3e7c1-f477-4c63-9b4d-da465369f449\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") "
Oct 30 20:58:05.386475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.386341    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-metrics-certs\") pod \"95a3e7c1-f477-4c63-9b4d-da465369f449\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") "
Oct 30 20:58:05.386475 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.386446    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95a3e7c1-f477-4c63-9b4d-da465369f449-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a3e7c1-f477-4c63-9b4d-da465369f449-openshift-service-ca\") pod \"95a3e7c1-f477-4c63-9b4d-da465369f449\" (UID: \"95a3e7c1-f477-4c63-9b4d-da465369f449\") "
Oct 30 20:58:05.386959 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.386929    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a3e7c1-f477-4c63-9b4d-da465369f449-openshift-service-ca" pod "95a3e7c1-f477-4c63-9b4d-da465369f449" (UID: "95a3e7c1-f477-4c63-9b4d-da465369f449"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:58:05.387201 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.387177    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4214ba5f-63d1-4448-abad-833ebbf18022-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.387289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.387206    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95a3e7c1-f477-4c63-9b4d-da465369f449-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.387289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.387222    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4214ba5f-63d1-4448-abad-833ebbf18022-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.387289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.387237    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.387289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.387250    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6dm6\" (UniqueName: \"kubernetes.io/projected/4214ba5f-63d1-4448-abad-833ebbf18022-kube-api-access-m6dm6\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.388809 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.388782    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-maas-api-tls" pod "95a3e7c1-f477-4c63-9b4d-da465369f449" (UID: "95a3e7c1-f477-4c63-9b4d-da465369f449"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:58:05.389098 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.389069    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a3e7c1-f477-4c63-9b4d-da465369f449-kube-api-access-dqcw7" pod "95a3e7c1-f477-4c63-9b4d-da465369f449" (UID: "95a3e7c1-f477-4c63-9b4d-da465369f449"). InnerVolumeSpecName "kube-api-access-dqcw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:58:05.389221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.389195    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-metrics-certs" pod "95a3e7c1-f477-4c63-9b4d-da465369f449" (UID: "95a3e7c1-f477-4c63-9b4d-da465369f449"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:58:05.391299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.391277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:05.493322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.492823    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.493322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.492874    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqcw7\" (UniqueName: \"kubernetes.io/projected/95a3e7c1-f477-4c63-9b4d-da465369f449-kube-api-access-dqcw7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.493322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.492893    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95a3e7c1-f477-4c63-9b4d-da465369f449-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:05.512552 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.502248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:05.596346 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.595062    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:05.628435 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.628400    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"]
Oct 30 20:58:05.633283 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.633252    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-bsp9q"]
Oct 30 20:58:05.746055 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.745954    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:05.837069 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:05.837022    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:06.013537 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.013504    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" path="/var/lib/kubelet/pods/4214ba5f-63d1-4448-abad-833ebbf18022/volumes"
Oct 30 20:58:06.264615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.264535    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"
Oct 30 20:58:06.264783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.264535    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk" event={"ID":"95a3e7c1-f477-4c63-9b4d-da465369f449","Type":"ContainerDied","Data":"f18d98f206d10d5cd62f86cf77e4c48edef2c825c67d0bece071e24fecfcd928"}
Oct 30 20:58:06.264783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.264676    4852 scope.go:122] "RemoveContainer" containerID="dd92e733df0ba7e818999196207dfe7bc2a88039d6fbd9c61c4a1e967f62a0d1"
Oct 30 20:58:06.285245 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.285219    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"]
Oct 30 20:58:06.292676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.292653    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5452c3-656b7d7496-c2txk"]
Oct 30 20:58:06.500791 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.500757    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"]
Oct 30 20:58:06.501418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501367    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f58c8df7-9da6-468d-aa6c-41531a2f31dc" containerName="maas-api"
Oct 30 20:58:06.501418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501404    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f58c8df7-9da6-468d-aa6c-41531a2f31dc" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501434    4852 cpu_manager.go:414] "containerMap: removing container" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501442    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501464    4852 cpu_manager.go:414] "containerMap: removing container" podUID="95a3e7c1-f477-4c63-9b4d-da465369f449" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501472    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95a3e7c1-f477-4c63-9b4d-da465369f449" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501488    4852 cpu_manager.go:414] "containerMap: removing container" podUID="4232a13e-0d80-4c2f-9497-0d5c944369a1" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501494    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4232a13e-0d80-4c2f-9497-0d5c944369a1" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501506    4852 cpu_manager.go:414] "containerMap: removing container" podUID="adba8e8d-5a5e-445f-b223-6c3f43160905" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501513    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="adba8e8d-5a5e-445f-b223-6c3f43160905" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501523    4852 cpu_manager.go:414] "containerMap: removing container" podUID="eb93b2e4-165f-4c52-b555-6d34f5b3df9c" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501530    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb93b2e4-165f-4c52-b555-6d34f5b3df9c" containerName="maas-api"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501549    4852 cpu_manager.go:414] "containerMap: removing container" podUID="a117995e-6ced-4d79-b4be-2e7604d0a8d6" containerName="payload-processing"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501559    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a117995e-6ced-4d79-b4be-2e7604d0a8d6" containerName="payload-processing"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501574    4852 cpu_manager.go:414] "containerMap: removing container" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy"
Oct 30 20:58:06.501593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501581    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="061e7d86-7f1c-40db-8ecd-76ce35bcadee" containerName="istio-proxy"
Oct 30 20:58:06.502242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501738    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="4214ba5f-63d1-4448-abad-833ebbf18022" containerName="istio-proxy"
Oct 30 20:58:06.502242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501754    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="95a3e7c1-f477-4c63-9b4d-da465369f449" containerName="maas-api"
Oct 30 20:58:06.502242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501765    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="4232a13e-0d80-4c2f-9497-0d5c944369a1" containerName="maas-api"
Oct 30 20:58:06.502242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501774    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="adba8e8d-5a5e-445f-b223-6c3f43160905" containerName="maas-api"
Oct 30 20:58:06.502242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501786    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb93b2e4-165f-4c52-b555-6d34f5b3df9c" containerName="maas-api"
Oct 30 20:58:06.502242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501798    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="a117995e-6ced-4d79-b4be-2e7604d0a8d6" containerName="payload-processing"
Oct 30 20:58:06.502242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.501813    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f58c8df7-9da6-468d-aa6c-41531a2f31dc" containerName="maas-api"
Oct 30 20:58:06.506994 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.506974    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.516028 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.515967    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-38c5ca-serving-cert\""
Oct 30 20:58:06.516478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.516216    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"]
Oct 30 20:58:06.536009 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.535978    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:06.615199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.615143    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.615486 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.615463    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.615679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.615647    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c9pr\" (UniqueName: \"kubernetes.io/projected/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-kube-api-access-4c9pr\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.615679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.615491    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:06.615864 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.615703    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-metrics-certs\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.718008 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.717969    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.718169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.718037    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.718169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.718069    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c9pr\" (UniqueName: \"kubernetes.io/projected/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-kube-api-access-4c9pr\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.718169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.718100    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-metrics-certs\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.718668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.718635    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.720666 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.720647    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.720770 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.720703    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-metrics-certs\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.729537 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.729512    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c9pr\" (UniqueName: \"kubernetes.io/projected/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-kube-api-access-4c9pr\") pod \"maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:06.781824 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.781743    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:06.829355 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:06.829328    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:07.113949 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.113921    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"]
Oct 30 20:58:07.114573 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:58:07.114540    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b7608d_41aa_46ca_b9b7_bbf20c268f7f.slice/crio-a4aa04d49b247555787112257713e143b45ed7257659811f220595d4444c5b3f WatchSource:0}: Error finding container a4aa04d49b247555787112257713e143b45ed7257659811f220595d4444c5b3f: Status 404 returned error can't find the container with id a4aa04d49b247555787112257713e143b45ed7257659811f220595d4444c5b3f
Oct 30 20:58:07.278305 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.278271    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx" event={"ID":"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f","Type":"ContainerStarted","Data":"a4aa04d49b247555787112257713e143b45ed7257659811f220595d4444c5b3f"}
Oct 30 20:58:07.305839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.305803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:07.633626 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.633578    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:07.695950 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.695919    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"]
Oct 30 20:58:07.700285 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.700260    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.708042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.707747    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-c72dc9-openshift-default-dockercfg-tzh8q\""
Oct 30 20:58:07.712428 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.712343    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"]
Oct 30 20:58:07.747705 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.747668    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844659    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c106f3-c79c-4881-8097-4e577f5a82f3-istiod-ca-cert\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844705    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-envoy\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844740    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-data\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844782    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-certs\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844816    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49rnm\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-kube-api-access-49rnm\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844844    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-token\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844888    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-credential-socket\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844920    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-podinfo\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.861576 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.844987    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-socket\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.946546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946510    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-token\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.946747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946560    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-credential-socket\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.946747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946587    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-podinfo\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.946747 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946632    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-socket\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.946935 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946828    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c106f3-c79c-4881-8097-4e577f5a82f3-istiod-ca-cert\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.946935 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946844    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-envoy\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.946935 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946867    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-data\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.947112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946895    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-certs\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.947112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.946984    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49rnm\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-kube-api-access-49rnm\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.947112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.947046    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-credential-socket\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.947112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.947080    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-socket\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.947309 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.947216    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-data\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.947370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.947306    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-certs\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.947447 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.947411    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c106f3-c79c-4881-8097-4e577f5a82f3-istiod-ca-cert\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.949007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.948985    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-envoy\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.949207 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.949191    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-podinfo\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.955273 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.955243    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-token\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:07.955955 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:07.955926    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49rnm\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-kube-api-access-49rnm\") pod \"e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") " pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:08.001169 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.001132    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:08.006655 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.006614    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95a3e7c1-f477-4c63-9b4d-da465369f449" path="/var/lib/kubelet/pods/95a3e7c1-f477-4c63-9b4d-da465369f449/volumes"
Oct 30 20:58:08.018620 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.018589    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:08.287198 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.287164    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx" event={"ID":"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f","Type":"ContainerStarted","Data":"4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41"}
Oct 30 20:58:08.287425 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.287277    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:08.306344 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.306292    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx" podStartSLOduration=1.8941515070000001 podStartE2EDuration="2.306277152s" totalImagesPullingTime="412.125645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:58:08.304529337 +0000 UTC m=+1196.419263528" watchObservedRunningTime="2026-10-30 20:58:08.306277152 +0000 UTC m=+1196.421011379"
Oct 30 20:58:08.352177 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.352141    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"]
Oct 30 20:58:08.352574 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:58:08.352546    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c106f3_c79c_4881_8097_4e577f5a82f3.slice/crio-99632afede219c5d3d2fb432e29a9c0b62fd0d33a14738f665804544ba22aec4 WatchSource:0}: Error finding container 99632afede219c5d3d2fb432e29a9c0b62fd0d33a14738f665804544ba22aec4: Status 404 returned error can't find the container with id 99632afede219c5d3d2fb432e29a9c0b62fd0d33a14738f665804544ba22aec4
Oct 30 20:58:08.355272 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.355242    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:58:08.355344 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.355307    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:58:08.355344 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.355336    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:58:08.579106 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:08.579046    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:09.296891 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:09.296853    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" event={"ID":"d8c106f3-c79c-4881-8097-4e577f5a82f3","Type":"ContainerStarted","Data":"41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe"}
Oct 30 20:58:09.297312 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:09.296910    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" event={"ID":"d8c106f3-c79c-4881-8097-4e577f5a82f3","Type":"ContainerStarted","Data":"99632afede219c5d3d2fb432e29a9c0b62fd0d33a14738f665804544ba22aec4"}
Oct 30 20:58:09.305840 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:09.305805    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:09.320449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:09.316693    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podStartSLOduration=2.316678711 podStartE2EDuration="2.316678711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:58:09.316590154 +0000 UTC m=+1197.431324348" watchObservedRunningTime="2026-10-30 20:58:09.316678711 +0000 UTC m=+1197.431412902"
Oct 30 20:58:10.018873 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:10.018842    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:10.024640 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:10.024617    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:10.040885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:10.040857    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:10.304299 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:10.304220    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:10.305203 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:10.305179    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:10.924771 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:10.924739    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:11.464583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:11.464552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:11.568686 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:11.568652    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:12.582460 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:12.582422    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:12.708371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:12.708293    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:12.805991 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:12.805961    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:13.281206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:13.281169    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:13.373567 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:13.373535    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:13.384944 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:13.384913    4852 scope.go:122] "RemoveContainer" containerID="e00c91d2e068eaab25a6d3f2deef5c299114c08164e487b9187239b7a7acb6f4"
Oct 30 20:58:13.405418 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:13.405210    4852 scope.go:122] "RemoveContainer" containerID="84bdbe70be4bf0299dcb7fcf04610bd62b35e757f2f3e88c667ea20a55bc7f4a"
Oct 30 20:58:13.613847 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:13.613765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:14.303379 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:14.303348    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:58:14.522266 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:14.522229    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:14.707631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:14.707538    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:15.118434 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:15.111733    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:15.240499 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:15.240468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:16.261653 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:16.261619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:17.480584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:17.480549    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:18.716687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:18.716653    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:19.068679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:19.068642    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:19.201630 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:19.201598    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:19.301474 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:19.301440    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:19.648897 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:19.648859    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:19.748357 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:19.748319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:20.474885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:20.474830    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:20.583448 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:20.583365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:20.683301 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:20.683258    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:20.835091 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:20.835057    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:21.095559 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:21.095478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:21.373367 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:21.373280    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:21.588645 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:21.588601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:21.624508 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:58:21.624375    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:58:21.826914 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:21.826882    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:22.231986 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:22.231953    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:22.364318 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:22.364282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:22.484908 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:22.484830    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:23.342468 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:23.342428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:24.245500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:24.245462    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:25.524641 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:25.524605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:25.625503 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:25.625465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:27.030120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:27.030086    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:27.123218 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:27.123186    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:27.232256 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:27.232224    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:28.425589 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:28.425528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:28.716432 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:28.716312    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:28.841040 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:28.841000    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:30.084584 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:30.084511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:30.184873 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:30.184840    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:30.355192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:30.355097    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:31.387478 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:31.387438    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:32.343511 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:32.343456    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:33.219902 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:33.219868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:34.109591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:34.109553    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:35.096242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:35.096210    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:35.930305 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:35.930274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:36.009109 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:36.009076    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:36.113692 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:36.113642    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:36.609251 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:36.609217    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:36.917360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:36.917286    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:37.436500 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:37.436464    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:37.700591 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:37.700496    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:37.804502 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:37.804461    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:37.987373 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:37.987266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:38.290306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:38.290161    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:38.529957 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:38.529923    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:38.817221 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:38.817181    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:39.229300 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:39.229218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:39.335638 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:39.335595    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:40.274060 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:40.274030    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:41.452152 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:41.452121    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:41.858427 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:41.858375    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"]
Oct 30 20:58:41.858744 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:41.858712    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy" containerID="cri-o://9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2" gracePeriod=30
Oct 30 20:58:41.986598 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:41.986564    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:42.092920 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:42.092880    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:42.189515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:42.189420    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:42.889515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:42.889484    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:43.262854 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:43.262811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:43.354829 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:43.354786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:44.280780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:44.280746    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:44.710896 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:44.710810    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:44.816928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:44.816894    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:45.725551 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:45.725520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:47.170641 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.146321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:47.359767 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.359730    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.432907    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-istio-token\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-istio-token\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433008    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/466e556f-2463-4275-a27f-23e56e536039-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466e556f-2463-4275-a27f-23e56e536039-istio-podinfo\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433062    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-envoy\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433117    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/466e556f-2463-4275-a27f-23e56e536039-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466e556f-2463-4275-a27f-23e56e536039-istiod-ca-cert\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433247    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-socket\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433321    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-credential-socket\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433511    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-data\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433545    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-certs\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.433625    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-kube-api-access-mhd7w\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-kube-api-access-mhd7w\") pod \"466e556f-2463-4275-a27f-23e56e536039\" (UID: \"466e556f-2463-4275-a27f-23e56e536039\") "
Oct 30 20:58:47.435129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.434823    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-data" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:47.435890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.435358    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-credential-socket" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:47.435890 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.435561    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-socket" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:47.436015 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.435905    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-certs" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:47.436313 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.436268    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/466e556f-2463-4275-a27f-23e56e536039-istiod-ca-cert" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:58:47.438725 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.438697    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-envoy" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:58:47.451712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.451575    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/466e556f-2463-4275-a27f-23e56e536039-istio-podinfo" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:58:47.451712 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.451684    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-istio-token" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:58:47.452961 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.452924    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-kube-api-access-mhd7w" pod "466e556f-2463-4275-a27f-23e56e536039" (UID: "466e556f-2463-4275-a27f-23e56e536039"). InnerVolumeSpecName "kube-api-access-mhd7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:58:47.535476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535441    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535472    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/466e556f-2463-4275-a27f-23e56e536039-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535485    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535494    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/466e556f-2463-4275-a27f-23e56e536039-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535502    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535511    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535519    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535529    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/466e556f-2463-4275-a27f-23e56e536039-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.535730 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.535537    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhd7w\" (UniqueName: \"kubernetes.io/projected/466e556f-2463-4275-a27f-23e56e536039-kube-api-access-mhd7w\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:58:47.640704 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.640666    4852 generic.go:356] "Generic (PLEG): container finished" podID="466e556f-2463-4275-a27f-23e56e536039" containerID="9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2" exitCode=0
Oct 30 20:58:47.640913 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.640708    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" event={"ID":"466e556f-2463-4275-a27f-23e56e536039","Type":"ContainerDied","Data":"9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2"}
Oct 30 20:58:47.640913 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.640775    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5" event={"ID":"466e556f-2463-4275-a27f-23e56e536039","Type":"ContainerDied","Data":"9c30a25efef1ac25e4ccaeba0f1e818a6a2250fe68d1100ffc8816240c245bd1"}
Oct 30 20:58:47.640913 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.640800    4852 scope.go:122] "RemoveContainer" containerID="9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2"
Oct 30 20:58:47.640913 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.640727    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"
Oct 30 20:58:47.681563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.681528    4852 scope.go:122] "RemoveContainer" containerID="9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2"
Oct 30 20:58:47.682534 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:58:47.681982    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2\": container with ID starting with 9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2 not found: ID does not exist" containerID="9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2"
Oct 30 20:58:47.682534 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.682024    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2"} err="failed to get container status \"9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2\": rpc error: code = NotFound desc = could not find container \"9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2\": container with ID starting with 9aa978a54ca9845fd01973b40886ea1141241cf03d4484aafa890797431bb2e2 not found: ID does not exist"
Oct 30 20:58:47.699531 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.699449    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"]
Oct 30 20:58:47.703667 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:47.703639    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-z5lm5"]
Oct 30 20:58:48.005206 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:48.005114    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="466e556f-2463-4275-a27f-23e56e536039" path="/var/lib/kubelet/pods/466e556f-2463-4275-a27f-23e56e536039/volumes"
Oct 30 20:58:48.294678 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:48.294625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:49.072410 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:49.068027    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:49.430915 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:49.426315    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:49.527306 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:49.527272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:49.712408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:49.712299    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:50.127702 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:50.127664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:51.261493 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:51.260998    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:51.581424 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:58:51.581326    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:58:52.470190 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:52.470116    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:53.523812 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:53.523774    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:54.629120 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:54.629075    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:55.741505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:55.741455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:55.748081 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:55.748040    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": read tcp 10.133.0.2:46594->10.133.0.116:15021: read: connection reset by peer" start-of-body=
Oct 30 20:58:55.748321 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:55.748267    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": read tcp 10.133.0.2:46594->10.133.0.116:15021: read: connection reset by peer"
Oct 30 20:58:55.841997 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:55.841965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:55.994166 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:55.994049    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:56.751259 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.751230    4852 generic.go:356] "Generic (PLEG): container finished" podID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerID="41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe" exitCode=0
Oct 30 20:58:56.751741 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.751309    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" event={"ID":"d8c106f3-c79c-4881-8097-4e577f5a82f3","Type":"ContainerDied","Data":"41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe"}
Oct 30 20:58:56.751796 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.751780    4852 scope.go:122] "RemoveContainer" containerID="41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe"
Oct 30 20:58:56.753381 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.753350    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:58:56.753531 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.753449    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:58:56.753531 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.753503    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 20:58:56.907112 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.907076    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"]
Oct 30 20:58:56.908210 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.908180    4852 cpu_manager.go:414] "containerMap: removing container" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy"
Oct 30 20:58:56.908375 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.908359    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy"
Oct 30 20:58:56.908711 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.908691    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="466e556f-2463-4275-a27f-23e56e536039" containerName="istio-proxy"
Oct 30 20:58:56.912939 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.912917    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:56.918829 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.918800    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-b46fb7-serving-cert\""
Oct 30 20:58:56.921582 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.921555    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"]
Oct 30 20:58:56.957639 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:56.957596    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:57.042762 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.042723    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73cd97e-3efc-4fb2-984e-b7d5386957c4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.043213 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.043183    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr7sm\" (UniqueName: \"kubernetes.io/projected/a73cd97e-3efc-4fb2-984e-b7d5386957c4-kube-api-access-lr7sm\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.043421 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.043368    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-metrics-certs\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.043746 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.043718    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.062825 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.059893    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:57.145457 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.145417    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.145648 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:58:57.145543    4852 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-serving-cert: secret "maas-api-e2e-ipp-a-b46fb7-serving-cert" not found
Oct 30 20:58:57.145648 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.145561    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73cd97e-3efc-4fb2-984e-b7d5386957c4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.145648 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:58:57.145623    4852 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls podName:a73cd97e-3efc-4fb2-984e-b7d5386957c4 nodeName:}" failed. No retries permitted until 2026-10-30 20:58:57.645600557 +0000 UTC m=+1245.760334753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls") pod "maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx" (UID: "a73cd97e-3efc-4fb2-984e-b7d5386957c4") : secret "maas-api-e2e-ipp-a-b46fb7-serving-cert" not found
Oct 30 20:58:57.145783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.145700    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr7sm\" (UniqueName: \"kubernetes.io/projected/a73cd97e-3efc-4fb2-984e-b7d5386957c4-kube-api-access-lr7sm\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.145783 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.145749    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-metrics-certs\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.146400 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.146356    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73cd97e-3efc-4fb2-984e-b7d5386957c4-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.149359 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.149328    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-metrics-certs\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.154076 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.154052    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr7sm\" (UniqueName: \"kubernetes.io/projected/a73cd97e-3efc-4fb2-984e-b7d5386957c4-kube-api-access-lr7sm\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.207481 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.207443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:57.675412 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.672503    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.715619 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.701558    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:57.784840 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.784806    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" event={"ID":"d8c106f3-c79c-4881-8097-4e577f5a82f3","Type":"ContainerStarted","Data":"58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389"}
Oct 30 20:58:57.844241 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:57.844198    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:58:58.015093 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.015059    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:58.019085 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.019055    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:58:58.019248 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.019216    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 20:58:58.019336 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.019258    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 20:58:58.174521 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.174479    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"]
Oct 30 20:58:58.176894 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:58:58.176819    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73cd97e_3efc_4fb2_984e_b7d5386957c4.slice/crio-582bb1a94ca43bf9557c8ce81515c8b0d539cf93f4042ba570438462f572a198 WatchSource:0}: Error finding container 582bb1a94ca43bf9557c8ce81515c8b0d539cf93f4042ba570438462f572a198: Status 404 returned error can't find the container with id 582bb1a94ca43bf9557c8ce81515c8b0d539cf93f4042ba570438462f572a198
Oct 30 20:58:58.178751 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.178729    4852 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:58:58.795466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.795422    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx" event={"ID":"a73cd97e-3efc-4fb2-984e-b7d5386957c4","Type":"ContainerStarted","Data":"aa3bb846bbddf283cb123fbda4e5fe07bbd37d42513f2cc34ef652061e74e7c4"}
Oct 30 20:58:58.795928 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.795525    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx" event={"ID":"a73cd97e-3efc-4fb2-984e-b7d5386957c4","Type":"ContainerStarted","Data":"582bb1a94ca43bf9557c8ce81515c8b0d539cf93f4042ba570438462f572a198"}
Oct 30 20:58:58.820049 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.820015    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"]
Oct 30 20:58:58.820289 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.820263    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy" containerID="cri-o://bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b" gracePeriod=30
Oct 30 20:58:58.821671 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.821621    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx" podStartSLOduration=2.359973938 podStartE2EDuration="2.821604047s" totalImagesPullingTime="461.630109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:58:58.815341552 +0000 UTC m=+1246.930075743" watchObservedRunningTime="2026-10-30 20:58:58.821604047 +0000 UTC m=+1246.936338240"
Oct 30 20:58:58.974563 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:58.949803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:59.019612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:59.019578    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 20:58:59.019780 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:59.019640    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 20:58:59.540960 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:59.540927    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:59.640749 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:59.640704    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:58:59.821469 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:58:59.820419    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:59:00.031006 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:00.030966    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 20:59:00.031211 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:00.031044    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 20:59:00.445876 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:00.445838    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:01.019191 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:01.019157    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 20:59:01.019676 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:01.019222    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 20:59:01.545199 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:01.545161    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:02.018994 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:02.018957    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Oct 30 20:59:02.019176 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:02.019019    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Oct 30 20:59:02.375594 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:02.375496    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:03.026114 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:03.026085    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:59:03.026527 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:03.026500    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:59:03.027362 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:03.027342    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:59:03.145466 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:03.145353    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:03.746426 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:03.746368    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:04.021176 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.021143    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:04.101354 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.101324    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:59:04.212885 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.212840    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-certs\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.212927    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-credential-socket\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.212960    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-socket\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213071 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213039    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/41a9f3ba-4151-4074-b25e-a12adb278946-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a9f3ba-4151-4074-b25e-a12adb278946-istiod-ca-cert\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213257 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213125    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/41a9f3ba-4151-4074-b25e-a12adb278946-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a9f3ba-4151-4074-b25e-a12adb278946-istio-podinfo\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213257 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213152    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-certs" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:04.213257 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213203    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-envoy\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213257 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213235    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-data\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213287    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-kube-api-access-qlg7q\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-kube-api-access-qlg7q\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213325    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-credential-socket" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:04.213515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213341    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-socket" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:04.213515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213377    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-istio-token\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-istio-token\") pod \"41a9f3ba-4151-4074-b25e-a12adb278946\" (UID: \"41a9f3ba-4151-4074-b25e-a12adb278946\") "
Oct 30 20:59:04.213745 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.213518    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-data" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:04.214135 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.214109    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.214135 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.214118    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a9f3ba-4151-4074-b25e-a12adb278946-istiod-ca-cert" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:59:04.214326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.214139    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.214326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.214155    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.214326 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.214170    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.215685 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.215661    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41a9f3ba-4151-4074-b25e-a12adb278946-istio-podinfo" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:59:04.215824 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.215673    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-kube-api-access-qlg7q" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "kube-api-access-qlg7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:59:04.215824 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.215794    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-envoy" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:04.216083 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.215898    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-istio-token" pod "41a9f3ba-4151-4074-b25e-a12adb278946" (UID: "41a9f3ba-4151-4074-b25e-a12adb278946"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:59:04.316099 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.316063    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41a9f3ba-4151-4074-b25e-a12adb278946-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.316099 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.316095    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41a9f3ba-4151-4074-b25e-a12adb278946-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.316315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.316110    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41a9f3ba-4151-4074-b25e-a12adb278946-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.316315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.316119    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlg7q\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-kube-api-access-qlg7q\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.316315 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.316131    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41a9f3ba-4151-4074-b25e-a12adb278946-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:04.827353 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.827315    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:04.857759 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.857721    4852 generic.go:356] "Generic (PLEG): container finished" podID="41a9f3ba-4151-4074-b25e-a12adb278946" containerID="bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b" exitCode=0
Oct 30 20:59:04.857924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.857770    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"
Oct 30 20:59:04.857924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.857804    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" event={"ID":"41a9f3ba-4151-4074-b25e-a12adb278946","Type":"ContainerDied","Data":"bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b"}
Oct 30 20:59:04.857924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.857859    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77" event={"ID":"41a9f3ba-4151-4074-b25e-a12adb278946","Type":"ContainerDied","Data":"45a8e3d35ffc62aa2b9bd566dbf70003cdd5530698e3855bfcd7dd06ffdc6767"}
Oct 30 20:59:04.857924 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.857889    4852 scope.go:122] "RemoveContainer" containerID="bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b"
Oct 30 20:59:04.869360 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.869338    4852 scope.go:122] "RemoveContainer" containerID="bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b"
Oct 30 20:59:04.869627 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:04.869610    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b\": container with ID starting with bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b not found: ID does not exist" containerID="bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b"
Oct 30 20:59:04.869679 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.869635    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b"} err="failed to get container status \"bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b\": rpc error: code = NotFound desc = could not find container \"bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b\": container with ID starting with bf000e14a2e6c0198e9e2ee4ae10428c69e840b1aa52b4a0ec3efa5e87558a1b not found: ID does not exist"
Oct 30 20:59:04.885408 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.885340    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"]
Oct 30 20:59:04.892429 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:04.892356    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-26f77"]
Oct 30 20:59:05.629361 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:05.629325    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:05.833462 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:05.833426    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 20:59:06.005440 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:06.005345    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" path="/var/lib/kubelet/pods/41a9f3ba-4151-4074-b25e-a12adb278946/volumes"
Oct 30 20:59:06.450753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:06.450667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:07.260125 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:07.260095    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:08.369030 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:08.368996    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:09.482546 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:09.482507    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:11.051871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:11.051832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:11.266214 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:11.266174    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:11.368580 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:11.368487    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:12.164022 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:12.163988    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:12.266439 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:12.266376    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:13.483839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:13.483809    4852 scope.go:122] "RemoveContainer" containerID="4bf9a0338bc441a9c16cf5aab0a024e7d6396f3a8ef05359671ea19120431de0"
Oct 30 20:59:13.493612 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:13.493589    4852 scope.go:122] "RemoveContainer" containerID="4b71fa914afe438d83ebacec1d0ac8e7e1016e2a0ddb65131e6ce61f060584c9"
Oct 30 20:59:13.840590 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:13.840559    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:15.835524 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:15.835493    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:16.389553 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:16.389518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:16.857482 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:16.857454    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:16.961989 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:16.959098    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:17.499639 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:17.499593    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:17.613898 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:17.613866    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:17.714269 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:17.714233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:17.864635 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:17.864597    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:18.833709 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:18.833678    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:20.010192 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:20.010157    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:20.612572 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:20.612518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:20.709900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:20.709872    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:21.228164 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.228135    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:21.557874 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:21.557838    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:59:21.839239 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.839151    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:21.925909 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.925877    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"]
Oct 30 20:59:21.926631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.926610    4852 cpu_manager.go:414] "containerMap: removing container" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy"
Oct 30 20:59:21.926631 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.926630    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy"
Oct 30 20:59:21.926803 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.926753    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="41a9f3ba-4151-4074-b25e-a12adb278946" containerName="istio-proxy"
Oct 30 20:59:21.929937 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.929914    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:21.930254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.930236    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:21.938757 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.938729    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-b80638-dockercfg-ncpdp\""
Oct 30 20:59:21.939032 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.939009    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-b80638\""
Oct 30 20:59:21.945850 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:21.945828    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"]
Oct 30 20:59:22.007111 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.007075    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2v8j\" (UniqueName: \"kubernetes.io/projected/239e4df8-993a-482c-9856-4f406a691d45-kube-api-access-f2v8j\") pod \"payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:22.007268 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.007153    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239e4df8-993a-482c-9856-4f406a691d45-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:22.109748 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.109653    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2v8j\" (UniqueName: \"kubernetes.io/projected/239e4df8-993a-482c-9856-4f406a691d45-kube-api-access-f2v8j\") pod \"payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:22.110007 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.109986    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239e4df8-993a-482c-9856-4f406a691d45-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:22.128129 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.127169    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239e4df8-993a-482c-9856-4f406a691d45-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:22.132899 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.132855    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2v8j\" (UniqueName: \"kubernetes.io/projected/239e4df8-993a-482c-9856-4f406a691d45-kube-api-access-f2v8j\") pod \"payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:22.167617 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.167580    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:22.251516 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.251476    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:22.540758 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.540729    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"]
Oct 30 20:59:22.542128 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:59:22.542091    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239e4df8_993a_482c_9856_4f406a691d45.slice/crio-472db3b4487aea1cc8c04364ea2eaec31b43ad106aa9e75fc131ab16f576238f WatchSource:0}: Error finding container 472db3b4487aea1cc8c04364ea2eaec31b43ad106aa9e75fc131ab16f576238f: Status 404 returned error can't find the container with id 472db3b4487aea1cc8c04364ea2eaec31b43ad106aa9e75fc131ab16f576238f
Oct 30 20:59:22.945596 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:22.945492    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:23.101042 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:23.101004    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6" event={"ID":"239e4df8-993a-482c-9856-4f406a691d45","Type":"ContainerStarted","Data":"0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e"}
Oct 30 20:59:23.101242 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:23.101067    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6" event={"ID":"239e4df8-993a-482c-9856-4f406a691d45","Type":"ContainerStarted","Data":"472db3b4487aea1cc8c04364ea2eaec31b43ad106aa9e75fc131ab16f576238f"}
Oct 30 20:59:23.101342 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:23.101246    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:23.121664 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:23.121618    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6" podStartSLOduration=2.121604707 podStartE2EDuration="2.121604707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:23.117684711 +0000 UTC m=+1271.232418913" watchObservedRunningTime="2026-10-30 20:59:23.121604707 +0000 UTC m=+1271.236338928"
Oct 30 20:59:23.784634 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:23.784590    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:23.969417 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:23.969054    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:24.077377 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:24.077293    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:24.162528 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:24.162494    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:24.338603 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:24.338521    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:24.662492 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:24.662376    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:25.332132 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:25.332096    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:26.209187 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.209150    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"]
Oct 30 20:59:26.209719 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.209684    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx" podUID="f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" containerName="maas-api" containerID="cri-o://4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41" gracePeriod=30
Oct 30 20:59:26.232116 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.232056    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:26.580755 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.580727    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:59:26.624344 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.624307    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-kube-api-access-4c9pr\" (UniqueName: \"kubernetes.io/projected/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-kube-api-access-4c9pr\") pod \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") "
Oct 30 20:59:26.624562 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.624483    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-maas-api-tls\") pod \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") "
Oct 30 20:59:26.624636 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.624621    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-openshift-service-ca\") pod \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") "
Oct 30 20:59:26.624700 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.624653    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-metrics-certs\") pod \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\" (UID: \"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f\") "
Oct 30 20:59:26.625061 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.625023    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-openshift-service-ca" pod "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" (UID: "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:59:26.625342 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.625318    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:26.626722 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.626689    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-kube-api-access-4c9pr" pod "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" (UID: "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f"). InnerVolumeSpecName "kube-api-access-4c9pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:59:26.626837 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.626794    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-maas-api-tls" pod "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" (UID: "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:59:26.627310 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.627292    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-metrics-certs" pod "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" (UID: "f1b7608d-41aa-46ca-b9b7-bbf20c268f7f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:59:26.726905 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.726870    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c9pr\" (UniqueName: \"kubernetes.io/projected/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-kube-api-access-4c9pr\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:26.726905 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.726903    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:26.726905 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.726913    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:26.795456 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.795343    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:26.915607 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:26.915568    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:27.023867 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.023833    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:27.172629 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.172535    4852 generic.go:356] "Generic (PLEG): container finished" podID="f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" containerID="4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41" exitCode=0
Oct 30 20:59:27.172629 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.172614    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"
Oct 30 20:59:27.172871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.172663    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx" event={"ID":"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f","Type":"ContainerDied","Data":"4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41"}
Oct 30 20:59:27.172871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.172720    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx" event={"ID":"f1b7608d-41aa-46ca-b9b7-bbf20c268f7f","Type":"ContainerDied","Data":"a4aa04d49b247555787112257713e143b45ed7257659811f220595d4444c5b3f"}
Oct 30 20:59:27.172871 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.172749    4852 scope.go:122] "RemoveContainer" containerID="4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41"
Oct 30 20:59:27.185615 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.185592    4852 scope.go:122] "RemoveContainer" containerID="4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41"
Oct 30 20:59:27.185973 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:27.185952    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41\": container with ID starting with 4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41 not found: ID does not exist" containerID="4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41"
Oct 30 20:59:27.186066 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.185982    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41"} err="failed to get container status \"4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41\": rpc error: code = NotFound desc = could not find container \"4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41\": container with ID starting with 4d7da86570a30a4f6dcf0950c657a5ea69b60fa73be4c80d19d2444b86257f41 not found: ID does not exist"
Oct 30 20:59:27.201047 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.201005    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"]
Oct 30 20:59:27.205041 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.205011    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-38c5ca-9948bdd8c-njvxx"]
Oct 30 20:59:27.414443 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:27.414371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:28.006013 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:28.005980    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" path="/var/lib/kubelet/pods/f1b7608d-41aa-46ca-b9b7-bbf20c268f7f/volumes"
Oct 30 20:59:28.614523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:28.614488    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:29.723560 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:29.723524    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:29.878092 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:29.878054    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:31.441505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:31.441465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:31.541052 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:31.541007    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:31.907938 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:31.907888    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:32.043539 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:32.043505    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:33.615887 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:33.615846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:34.150952 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:34.150925    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:34.442642 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:34.442564    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:34.543115 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:34.543083    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:34.799753 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:34.799723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:34.971409 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:34.968674    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:35.723505 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:35.723462    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:36.382030 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.381994    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"]
Oct 30 20:59:36.382816 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.382793    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" containerName="maas-api"
Oct 30 20:59:36.382921 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.382816    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" containerName="maas-api"
Oct 30 20:59:36.383010 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.382996    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1b7608d-41aa-46ca-b9b7-bbf20c268f7f" containerName="maas-api"
Oct 30 20:59:36.388210 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.388189    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.392593 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.392551    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-b80638-serving-cert\""
Oct 30 20:59:36.399449 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.399422    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"]
Oct 30 20:59:36.469397 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.469350    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:36.476987 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.476959    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-metrics-certs\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.477128 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.477043    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f9c688-40a7-45f0-b51b-7691f9f3b07c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.477181 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.477126    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.477288 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.477267    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89lqt\" (UniqueName: \"kubernetes.io/projected/44f9c688-40a7-45f0-b51b-7691f9f3b07c-kube-api-access-89lqt\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.572098 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.572028    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:36.578774 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.578742    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89lqt\" (UniqueName: \"kubernetes.io/projected/44f9c688-40a7-45f0-b51b-7691f9f3b07c-kube-api-access-89lqt\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.578964 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.578870    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-metrics-certs\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.578964 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.578921    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f9c688-40a7-45f0-b51b-7691f9f3b07c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.578964 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.578958    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.579121 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:36.579106    4852 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-serving-cert: secret "maas-api-e2e-ipp-b-b80638-serving-cert" not found
Oct 30 20:59:36.579186 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:36.579177    4852 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls podName:44f9c688-40a7-45f0-b51b-7691f9f3b07c nodeName:}" failed. No retries permitted until 2026-10-30 20:59:37.079159656 +0000 UTC m=+1285.193893842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls") pod "maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g" (UID: "44f9c688-40a7-45f0-b51b-7691f9f3b07c") : secret "maas-api-e2e-ipp-b-b80638-serving-cert" not found
Oct 30 20:59:36.579696 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.579672    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f9c688-40a7-45f0-b51b-7691f9f3b07c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.581632 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.581607    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-metrics-certs\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.587891 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.587865    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89lqt\" (UniqueName: \"kubernetes.io/projected/44f9c688-40a7-45f0-b51b-7691f9f3b07c-kube-api-access-89lqt\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:36.680810 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:36.680714    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:37.095293 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:37.095261    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:37.097695 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:37.097669    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:37.305607 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:37.305569    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:37.318843 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:37.318815    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:37.650754 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:37.650718    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"]
Oct 30 20:59:37.651275 ip-10-0-135-178 kubenswrapper[4852]: W1030 20:59:37.651250    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f9c688_40a7_45f0_b51b_7691f9f3b07c.slice/crio-b555edd50d617cc3df7d8909dd5c91cf1f4272dd4d053d35d68cb29a2225b914 WatchSource:0}: Error finding container b555edd50d617cc3df7d8909dd5c91cf1f4272dd4d053d35d68cb29a2225b914: Status 404 returned error can't find the container with id b555edd50d617cc3df7d8909dd5c91cf1f4272dd4d053d35d68cb29a2225b914
Oct 30 20:59:38.049682 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.049655    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:38.235415 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.234133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:38.294464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.290303    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g" event={"ID":"44f9c688-40a7-45f0-b51b-7691f9f3b07c","Type":"ContainerStarted","Data":"a9519ef9616567bde06ec3032c0fc2d7377e8151ae9dd9a21173e1899f3f3834"}
Oct 30 20:59:38.294464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.290373    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g" event={"ID":"44f9c688-40a7-45f0-b51b-7691f9f3b07c","Type":"ContainerStarted","Data":"b555edd50d617cc3df7d8909dd5c91cf1f4272dd4d053d35d68cb29a2225b914"}
Oct 30 20:59:38.294464 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.290808    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:38.340454 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.329569    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g" podStartSLOduration=1.9474860029999999 podStartE2EDuration="2.329548571s" totalImagesPullingTime="382.062568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:59:38.327040184 +0000 UTC m=+1286.441774376" watchObservedRunningTime="2026-10-30 20:59:38.329548571 +0000 UTC m=+1286.444282762"
Oct 30 20:59:38.406523 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.406491    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:38.524663 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.524617    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:38.966929 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:38.966897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:39.401981 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:39.401946    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:40.303172 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:40.303133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:41.325151 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:41.325116    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:42.109884 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:42.109846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:42.603587 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:42.603546    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:42.708356 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:42.708283    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:42.838498 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:42.838457    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:43.718565 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:43.718529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:44.303496 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:44.303464    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 20:59:44.540623 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:44.540592    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:45.376476 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:45.376444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:45.478525 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:45.478494    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:45.602205 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:45.602167    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:45.687064 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:45.686972    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"]
Oct 30 20:59:45.687254 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:45.687231    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" containerID="cri-o://58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389" gracePeriod=30
Oct 30 20:59:45.732495 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:45.732458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:46.846304 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:46.846270    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:47.844414 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:47.844367    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:48.029322 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:48.029283    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 30 20:59:48.029715 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:48.029344    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 20:59:49.038814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:49.038776    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:49.647134 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:49.647095    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:50.254569 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:50.254532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:50.354512 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:50.354437    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:50.529465 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:50.529430    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:50.714149 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:50.714112    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:50.812775 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:50.812745    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:51.015204 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.015140    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:59:51.051883 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.051223    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"]
Oct 30 20:59:51.106227 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.106105    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:51.112049 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.112023    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-envoy\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.112318 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.112296    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-socket\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.112549 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.112532    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8c106f3-c79c-4881-8097-4e577f5a82f3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c106f3-c79c-4881-8097-4e577f5a82f3-istiod-ca-cert\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.112691 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.112675    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-credential-socket\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.112836 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.112820    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-podinfo\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.112972 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.112956    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-data\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.113193 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.113175    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-certs\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.113417 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.113398    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-token\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-token\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.113552 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.113532    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-kube-api-access-49rnm\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-kube-api-access-49rnm\") pod \"d8c106f3-c79c-4881-8097-4e577f5a82f3\" (UID: \"d8c106f3-c79c-4881-8097-4e577f5a82f3\") "
Oct 30 20:59:51.114605 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.113181    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-socket" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:51.114724 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.113536    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-credential-socket" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:51.114786 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.113897    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c106f3-c79c-4881-8097-4e577f5a82f3-istiod-ca-cert" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:59:51.114839 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.114146    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-certs" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:51.119601 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.119573    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-token" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:59:51.119940 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.119916    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-podinfo" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:59:51.120160 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.120139    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-data" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:51.135819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.135784    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-kube-api-access-49rnm" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "kube-api-access-49rnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:59:51.136684 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.136659    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-envoy" pod "d8c106f3-c79c-4881-8097-4e577f5a82f3" (UID: "d8c106f3-c79c-4881-8097-4e577f5a82f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:59:51.204520 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.204483    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:51.215978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.215950    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.215978 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.215979    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49rnm\" (UniqueName: \"kubernetes.io/projected/d8c106f3-c79c-4881-8097-4e577f5a82f3-kube-api-access-49rnm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.216240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.215995    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.216240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.216008    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.216240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.216020    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c106f3-c79c-4881-8097-4e577f5a82f3-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.216240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.216031    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.216240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.216044    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.216240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.216057    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.216240 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.216070    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c106f3-c79c-4881-8097-4e577f5a82f3-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:51.303956 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.303923    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:51.393687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.393601    4852 generic.go:356] "Generic (PLEG): container finished" podID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerID="58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389" exitCode=0
Oct 30 20:59:51.393687 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.393652    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"
Oct 30 20:59:51.393880 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.393687    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" event={"ID":"d8c106f3-c79c-4881-8097-4e577f5a82f3","Type":"ContainerDied","Data":"58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389"}
Oct 30 20:59:51.393880 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.393744    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt" event={"ID":"d8c106f3-c79c-4881-8097-4e577f5a82f3","Type":"ContainerDied","Data":"99632afede219c5d3d2fb432e29a9c0b62fd0d33a14738f665804544ba22aec4"}
Oct 30 20:59:51.393880 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.393772    4852 scope.go:122] "RemoveContainer" containerID="58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389"
Oct 30 20:59:51.405645 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.405626    4852 scope.go:122] "RemoveContainer" containerID="41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe"
Oct 30 20:59:51.406794 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.406775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:51.416370 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.416344    4852 scope.go:122] "RemoveContainer" containerID="58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389"
Oct 30 20:59:51.416728 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:51.416706    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389\": container with ID starting with 58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389 not found: ID does not exist" containerID="58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389"
Oct 30 20:59:51.416819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.416736    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389"} err="failed to get container status \"58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389\": rpc error: code = NotFound desc = could not find container \"58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389\": container with ID starting with 58467ff75f7d89d5faaba7f9ad2cde5977c42647e8d4e14488dd3c2d18647389 not found: ID does not exist"
Oct 30 20:59:51.416819 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.416754    4852 scope.go:122] "RemoveContainer" containerID="41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe"
Oct 30 20:59:51.417042 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:51.417024    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe\": container with ID starting with 41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe not found: ID does not exist" containerID="41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe"
Oct 30 20:59:51.417083 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.417051    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe"} err="failed to get container status \"41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe\": rpc error: code = NotFound desc = could not find container \"41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe\": container with ID starting with 41571ebdba092d824c70439952d31b8eb2683a128c26a93de32a49c2b55945fe not found: ID does not exist"
Oct 30 20:59:51.432515 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.432474    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"]
Oct 30 20:59:51.436371 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.436341    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-c72dc9-openshift-default-5b99fdc949-ghzjt"]
Oct 30 20:59:51.582411 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.581621    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:51.619674 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:51.619638    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 20:59:51.821258 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.821226    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:51.995600 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:51.995565    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:52.006663 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:52.006626    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" path="/var/lib/kubelet/pods/d8c106f3-c79c-4881-8097-4e577f5a82f3/volumes"
Oct 30 20:59:52.380900 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:52.380822    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:52.727814 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:52.727735    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:52.876485 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:52.876446    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:53.204812 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:53.204771    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:53.303606 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:53.303573    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:53.685286 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:53.685252    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:54.360737 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:54.360698    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:54.455257 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:54.455225    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:55.428540 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:55.428507    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:55.636001 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:55.635972    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:55.748236 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:55.748153    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:56.051668 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.051603    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6" podUID="239e4df8-993a-482c-9856-4f406a691d45" containerName="payload-pre-processing" containerID="cri-o://0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e" gracePeriod=25
Oct 30 20:59:56.349817 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.349789    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:56.434602 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.434552    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/239e4df8-993a-482c-9856-4f406a691d45-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239e4df8-993a-482c-9856-4f406a691d45-plugins-config-volume\") pod \"239e4df8-993a-482c-9856-4f406a691d45\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") "
Oct 30 20:59:56.435090 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.434623    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/239e4df8-993a-482c-9856-4f406a691d45-kube-api-access-f2v8j\" (UniqueName: \"kubernetes.io/projected/239e4df8-993a-482c-9856-4f406a691d45-kube-api-access-f2v8j\") pod \"239e4df8-993a-482c-9856-4f406a691d45\" (UID: \"239e4df8-993a-482c-9856-4f406a691d45\") "
Oct 30 20:59:56.435622 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.435589    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239e4df8-993a-482c-9856-4f406a691d45-plugins-config-volume" pod "239e4df8-993a-482c-9856-4f406a691d45" (UID: "239e4df8-993a-482c-9856-4f406a691d45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:59:56.437651 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.437620    4852 generic.go:356] "Generic (PLEG): container finished" podID="239e4df8-993a-482c-9856-4f406a691d45" containerID="0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e" exitCode=0
Oct 30 20:59:56.438621 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.437688    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"
Oct 30 20:59:56.438621 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.437709    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6" event={"ID":"239e4df8-993a-482c-9856-4f406a691d45","Type":"ContainerDied","Data":"0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e"}
Oct 30 20:59:56.438621 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.438426    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6" event={"ID":"239e4df8-993a-482c-9856-4f406a691d45","Type":"ContainerDied","Data":"472db3b4487aea1cc8c04364ea2eaec31b43ad106aa9e75fc131ab16f576238f"}
Oct 30 20:59:56.438621 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.438470    4852 scope.go:122] "RemoveContainer" containerID="0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e"
Oct 30 20:59:56.458038 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.457987    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239e4df8-993a-482c-9856-4f406a691d45-kube-api-access-f2v8j" pod "239e4df8-993a-482c-9856-4f406a691d45" (UID: "239e4df8-993a-482c-9856-4f406a691d45"). InnerVolumeSpecName "kube-api-access-f2v8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:59:56.458153 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.458134    4852 scope.go:122] "RemoveContainer" containerID="0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e"
Oct 30 20:59:56.458832 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.458633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:56.458832 ip-10-0-135-178 kubenswrapper[4852]: E1030 20:59:56.458756    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e\": container with ID starting with 0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e not found: ID does not exist" containerID="0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e"
Oct 30 20:59:56.458832 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.458800    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e"} err="failed to get container status \"0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e\": rpc error: code = NotFound desc = could not find container \"0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e\": container with ID starting with 0f1274851974d39771827ae4b17af6c2937522e31f99b4816a9fb60dea31b08e not found: ID does not exist"
Oct 30 20:59:56.536225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.536190    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239e4df8-993a-482c-9856-4f406a691d45-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:56.536225 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.536215    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2v8j\" (UniqueName: \"kubernetes.io/projected/239e4df8-993a-482c-9856-4f406a691d45-kube-api-access-f2v8j\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:59:56.546228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.546197    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:56.645979 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.645886    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:56.800228 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.800187    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"]
Oct 30 20:59:56.808816 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:56.808782    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-b80638-5948d6f78c-2twt6"]
Oct 30 20:59:57.597058 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:57.597025    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:58.005583 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:58.005555    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="239e4df8-993a-482c-9856-4f406a691d45" path="/var/lib/kubelet/pods/239e4df8-993a-482c-9856-4f406a691d45/volumes"
Oct 30 20:59:58.425159 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:58.425077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:59.165059 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:59.165022    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:59.259067 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:59.259031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:59.550284 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:59.550246    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 20:59:59.670235 ip-10-0-135-178 kubenswrapper[4852]: I1030 20:59:59.670199    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:00.147547 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.147515    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"]
Oct 30 21:00:00.148164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148144    4852 cpu_manager.go:414] "containerMap: removing container" podUID="239e4df8-993a-482c-9856-4f406a691d45" containerName="payload-pre-processing"
Oct 30 21:00:00.148164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148165    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="239e4df8-993a-482c-9856-4f406a691d45" containerName="payload-pre-processing"
Oct 30 21:00:00.148311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148188    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy"
Oct 30 21:00:00.148311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148194    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy"
Oct 30 21:00:00.148311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148204    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy"
Oct 30 21:00:00.148311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148209    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy"
Oct 30 21:00:00.148472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148322    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="239e4df8-993a-482c-9856-4f406a691d45" containerName="payload-pre-processing"
Oct 30 21:00:00.148472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148334    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy"
Oct 30 21:00:00.148472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.148341    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8c106f3-c79c-4881-8097-4e577f5a82f3" containerName="istio-proxy"
Oct 30 21:00:00.151972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.151956    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.178135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.178104    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"]
Oct 30 21:00:00.185936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.185083    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"]
Oct 30 21:00:00.211408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.211363    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.216684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.216483    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c9ee2-e911-4580-abc1-acdb42356ecc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.216684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.216657    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcrfb\" (UniqueName: \"kubernetes.io/projected/bf0c9ee2-e911-4580-abc1-acdb42356ecc-kube-api-access-dcrfb\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.221760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.221507    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"]
Oct 30 21:00:00.228961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.228147    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"]
Oct 30 21:00:00.236510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.236482    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.240988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.240693    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 21:00:00.248812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.248783    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"]
Oct 30 21:00:00.276558 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.276511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:00.318424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.318377    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d082bc30-f100-4dfd-99f8-476dbb765fed-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.318669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.318631    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcrfb\" (UniqueName: \"kubernetes.io/projected/bf0c9ee2-e911-4580-abc1-acdb42356ecc-kube-api-access-dcrfb\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.318782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.318673    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md6p4\" (UniqueName: \"kubernetes.io/projected/e6af05da-c933-443a-a0a2-3877af1dc2ff-kube-api-access-md6p4\") pod \"maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.318840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.318778    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6af05da-c933-443a-a0a2-3877af1dc2ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.318932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.318917    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjgk9\" (UniqueName: \"kubernetes.io/projected/d082bc30-f100-4dfd-99f8-476dbb765fed-kube-api-access-zjgk9\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.318979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.318944    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c9ee2-e911-4580-abc1-acdb42356ecc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.319666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.319640    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c9ee2-e911-4580-abc1-acdb42356ecc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.327424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.327381    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcrfb\" (UniqueName: \"kubernetes.io/projected/bf0c9ee2-e911-4580-abc1-acdb42356ecc-kube-api-access-dcrfb\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.373323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.373287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:00.419995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.419910    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d082bc30-f100-4dfd-99f8-476dbb765fed-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.419995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.419957    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md6p4\" (UniqueName: \"kubernetes.io/projected/e6af05da-c933-443a-a0a2-3877af1dc2ff-kube-api-access-md6p4\") pod \"maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.420241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.420021    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6af05da-c933-443a-a0a2-3877af1dc2ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.420241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.420144    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjgk9\" (UniqueName: \"kubernetes.io/projected/d082bc30-f100-4dfd-99f8-476dbb765fed-kube-api-access-zjgk9\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.420643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.420618    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d082bc30-f100-4dfd-99f8-476dbb765fed-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.420725 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.420708    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6af05da-c933-443a-a0a2-3877af1dc2ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.430911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.430886    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md6p4\" (UniqueName: \"kubernetes.io/projected/e6af05da-c933-443a-a0a2-3877af1dc2ff-kube-api-access-md6p4\") pod \"maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.431311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.431293    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjgk9\" (UniqueName: \"kubernetes.io/projected/d082bc30-f100-4dfd-99f8-476dbb765fed-kube-api-access-zjgk9\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.465123 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.465087    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:00.580411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.555517    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:00.580411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.565414    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:00.877584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:00.877544    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"]
Oct 30 21:00:00.877723 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:00:00.877625    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0c9ee2_e911_4580_abc1_acdb42356ecc.slice/crio-a18248dab2362a0f7f71f22ca580b496f2b54d5ba7d1f01b372c52e1ef185861 WatchSource:0}: Error finding container a18248dab2362a0f7f71f22ca580b496f2b54d5ba7d1f01b372c52e1ef185861: Status 404 returned error can't find the container with id a18248dab2362a0f7f71f22ca580b496f2b54d5ba7d1f01b372c52e1ef185861
Oct 30 21:00:01.015230 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.015195    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"]
Oct 30 21:00:01.015621 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:00:01.015595    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd082bc30_f100_4dfd_99f8_476dbb765fed.slice/crio-a224c835215484942f35958e797f734cee207ed1a2264b3f1ba85f552a4d04b4 WatchSource:0}: Error finding container a224c835215484942f35958e797f734cee207ed1a2264b3f1ba85f552a4d04b4: Status 404 returned error can't find the container with id a224c835215484942f35958e797f734cee207ed1a2264b3f1ba85f552a4d04b4
Oct 30 21:00:01.027503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.027479    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"]
Oct 30 21:00:01.028873 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:00:01.028847    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6af05da_c933_443a_a0a2_3877af1dc2ff.slice/crio-beaa307f28009205be10fae3a1c5f5efc2dd913bbab16cc44afd1d334dd14530 WatchSource:0}: Error finding container beaa307f28009205be10fae3a1c5f5efc2dd913bbab16cc44afd1d334dd14530: Status 404 returned error can't find the container with id beaa307f28009205be10fae3a1c5f5efc2dd913bbab16cc44afd1d334dd14530
Oct 30 21:00:01.107104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.107070    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:01.481125 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.481093    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" event={"ID":"bf0c9ee2-e911-4580-abc1-acdb42356ecc","Type":"ContainerStarted","Data":"29a10a14fe967cf2597f61abedd5e29a5ad1880ff2871d2060635f48d226a8a4"}
Oct 30 21:00:01.481589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.481144    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" event={"ID":"bf0c9ee2-e911-4580-abc1-acdb42356ecc","Type":"ContainerStarted","Data":"a18248dab2362a0f7f71f22ca580b496f2b54d5ba7d1f01b372c52e1ef185861"}
Oct 30 21:00:01.482957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.482932    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerStarted","Data":"0a134d612fbb7bb918a7d3a38f49f5fa5377156c130154eb02db1115a12672f6"}
Oct 30 21:00:01.483051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.482980    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerStarted","Data":"beaa307f28009205be10fae3a1c5f5efc2dd913bbab16cc44afd1d334dd14530"}
Oct 30 21:00:01.484512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.484488    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerStarted","Data":"90020fd2a28bfc192586409e83bc410c56eb76bcfd7391c5b7ab31401209dbf1"}
Oct 30 21:00:01.484610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.484528    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerStarted","Data":"a224c835215484942f35958e797f734cee207ed1a2264b3f1ba85f552a4d04b4"}
Oct 30 21:00:01.496216 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.496164    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" podStartSLOduration=1.496148524 podStartE2EDuration="1.496148524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:01.495594278 +0000 UTC m=+1309.610328468" watchObservedRunningTime="2026-10-30 21:00:01.496148524 +0000 UTC m=+1309.610882719"
Oct 30 21:00:01.513985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.513932    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" podStartSLOduration=1.5139190139999998 podStartE2EDuration="1.513919014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:01.509923336 +0000 UTC m=+1309.624657525" watchObservedRunningTime="2026-10-30 21:00:01.513919014 +0000 UTC m=+1309.628653203"
Oct 30 21:00:01.529724 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.529614    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" podStartSLOduration=1.5296010930000001 podStartE2EDuration="1.529601093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:01.526660132 +0000 UTC m=+1309.641394316" watchObservedRunningTime="2026-10-30 21:00:01.529601093 +0000 UTC m=+1309.644335319"
Oct 30 21:00:01.694920 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.694866    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:01.782266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:01.782166    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:02.025217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.025176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:02.108806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.108713    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:02.501769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.501688    4852 generic.go:356] "Generic (PLEG): container finished" podID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerID="0a134d612fbb7bb918a7d3a38f49f5fa5377156c130154eb02db1115a12672f6" exitCode=60
Oct 30 21:00:02.502266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.501772    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerDied","Data":"0a134d612fbb7bb918a7d3a38f49f5fa5377156c130154eb02db1115a12672f6"}
Oct 30 21:00:02.502266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.502074    4852 scope.go:122] "RemoveContainer" containerID="0a134d612fbb7bb918a7d3a38f49f5fa5377156c130154eb02db1115a12672f6"
Oct 30 21:00:02.503581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.503556    4852 generic.go:356] "Generic (PLEG): container finished" podID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerID="90020fd2a28bfc192586409e83bc410c56eb76bcfd7391c5b7ab31401209dbf1" exitCode=60
Oct 30 21:00:02.503709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.503631    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerDied","Data":"90020fd2a28bfc192586409e83bc410c56eb76bcfd7391c5b7ab31401209dbf1"}
Oct 30 21:00:02.503904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.503886    4852 scope.go:122] "RemoveContainer" containerID="90020fd2a28bfc192586409e83bc410c56eb76bcfd7391c5b7ab31401209dbf1"
Oct 30 21:00:02.505523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.505442    4852 generic.go:356] "Generic (PLEG): container finished" podID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerID="29a10a14fe967cf2597f61abedd5e29a5ad1880ff2871d2060635f48d226a8a4" exitCode=60
Oct 30 21:00:02.505711 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.505613    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" event={"ID":"bf0c9ee2-e911-4580-abc1-acdb42356ecc","Type":"ContainerDied","Data":"29a10a14fe967cf2597f61abedd5e29a5ad1880ff2871d2060635f48d226a8a4"}
Oct 30 21:00:02.505952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:02.505934    4852 scope.go:122] "RemoveContainer" containerID="29a10a14fe967cf2597f61abedd5e29a5ad1880ff2871d2060635f48d226a8a4"
Oct 30 21:00:03.322813 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.322722    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:03.515967 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.515936    4852 generic.go:356] "Generic (PLEG): container finished" podID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerID="7917932b06e80fccf5430e21c3ffcbced7eba65ba0fcc6450641dbf197daa2ad" exitCode=60
Oct 30 21:00:03.516471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.516018    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" event={"ID":"bf0c9ee2-e911-4580-abc1-acdb42356ecc","Type":"ContainerDied","Data":"7917932b06e80fccf5430e21c3ffcbced7eba65ba0fcc6450641dbf197daa2ad"}
Oct 30 21:00:03.516471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.516098    4852 scope.go:122] "RemoveContainer" containerID="29a10a14fe967cf2597f61abedd5e29a5ad1880ff2871d2060635f48d226a8a4"
Oct 30 21:00:03.516471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.516458    4852 scope.go:122] "RemoveContainer" containerID="7917932b06e80fccf5430e21c3ffcbced7eba65ba0fcc6450641dbf197daa2ad"
Oct 30 21:00:03.516868 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:03.516832    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq_odh-ai-gateway-infra(bf0c9ee2-e911-4580-abc1-acdb42356ecc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc"
Oct 30 21:00:03.518694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.518671    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerStarted","Data":"6ed609945684b51a449b4c27fc5e189ced507caef9729d1affcce30a141d1180"}
Oct 30 21:00:03.521182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.521157    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerStarted","Data":"453fe2b7080d4d6e93c7bf4da6d6da7318adea008d9cdcf5bd0671aa420f155a"}
Oct 30 21:00:03.873417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:03.873298    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:04.345854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.345820    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:04.429716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.429681    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:04.515717 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.515684    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:04.546621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.546591    4852 scope.go:122] "RemoveContainer" containerID="7917932b06e80fccf5430e21c3ffcbced7eba65ba0fcc6450641dbf197daa2ad"
Oct 30 21:00:04.549539 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:04.548983    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq_odh-ai-gateway-infra(bf0c9ee2-e911-4580-abc1-acdb42356ecc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc"
Oct 30 21:00:04.554410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.550564    4852 generic.go:356] "Generic (PLEG): container finished" podID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerID="6ed609945684b51a449b4c27fc5e189ced507caef9729d1affcce30a141d1180" exitCode=60
Oct 30 21:00:04.554410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.550752    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerDied","Data":"6ed609945684b51a449b4c27fc5e189ced507caef9729d1affcce30a141d1180"}
Oct 30 21:00:04.554410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.550816    4852 scope.go:122] "RemoveContainer" containerID="0a134d612fbb7bb918a7d3a38f49f5fa5377156c130154eb02db1115a12672f6"
Oct 30 21:00:04.554410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.551322    4852 scope.go:122] "RemoveContainer" containerID="6ed609945684b51a449b4c27fc5e189ced507caef9729d1affcce30a141d1180"
Oct 30 21:00:04.554410 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:04.551727    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5_odh-ai-gateway-infra(e6af05da-c933-443a-a0a2-3877af1dc2ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff"
Oct 30 21:00:04.555276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.554973    4852 generic.go:356] "Generic (PLEG): container finished" podID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerID="453fe2b7080d4d6e93c7bf4da6d6da7318adea008d9cdcf5bd0671aa420f155a" exitCode=60
Oct 30 21:00:04.555276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.555012    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerDied","Data":"453fe2b7080d4d6e93c7bf4da6d6da7318adea008d9cdcf5bd0671aa420f155a"}
Oct 30 21:00:04.561865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.555465    4852 scope.go:122] "RemoveContainer" containerID="453fe2b7080d4d6e93c7bf4da6d6da7318adea008d9cdcf5bd0671aa420f155a"
Oct 30 21:00:04.561865 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:04.555904    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt_odh-ai-gateway-infra(d082bc30-f100-4dfd-99f8-476dbb765fed)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed"
Oct 30 21:00:04.572417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:04.572382    4852 scope.go:122] "RemoveContainer" containerID="90020fd2a28bfc192586409e83bc410c56eb76bcfd7391c5b7ab31401209dbf1"
Oct 30 21:00:05.198589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:05.198532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:05.392411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:05.390420    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:05.499886 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:05.499792    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:05.565703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:05.565662    4852 scope.go:122] "RemoveContainer" containerID="6ed609945684b51a449b4c27fc5e189ced507caef9729d1affcce30a141d1180"
Oct 30 21:00:05.566110 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:05.566049    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5_odh-ai-gateway-infra(e6af05da-c933-443a-a0a2-3877af1dc2ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff"
Oct 30 21:00:05.568115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:05.568086    4852 scope.go:122] "RemoveContainer" containerID="453fe2b7080d4d6e93c7bf4da6d6da7318adea008d9cdcf5bd0671aa420f155a"
Oct 30 21:00:05.568465 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:05.568436    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt_odh-ai-gateway-infra(d082bc30-f100-4dfd-99f8-476dbb765fed)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed"
Oct 30 21:00:05.659708 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:05.659672    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:05.974127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:05.974074    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:07.121826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:07.121728    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:07.932461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:07.932422    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:08.846044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:08.846002    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:09.591789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:09.591744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:10.343529 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:10.343499    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:11.236279 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:11.236240    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:12.031103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:12.031073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:12.265673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:12.265639    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:12.366835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:12.366724    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:12.591090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:12.591059    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:13.037078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:13.037028    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:13.914894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:13.914817    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:14.973017 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:14.972984    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:15.997596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:15.997565    4852 scope.go:122] "RemoveContainer" containerID="453fe2b7080d4d6e93c7bf4da6d6da7318adea008d9cdcf5bd0671aa420f155a"
Oct 30 21:00:16.113658 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:16.113617    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:16.659905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:16.659825    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerStarted","Data":"1d0b5c8ac393d03bc57d69938c71a12caa211e0cdeb8816a5233a09c58eaa996"}
Oct 30 21:00:16.770044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:16.770004    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:16.998065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:16.997978    4852 scope.go:122] "RemoveContainer" containerID="7917932b06e80fccf5430e21c3ffcbced7eba65ba0fcc6450641dbf197daa2ad"
Oct 30 21:00:16.998557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:16.998280    4852 scope.go:122] "RemoveContainer" containerID="6ed609945684b51a449b4c27fc5e189ced507caef9729d1affcce30a141d1180"
Oct 30 21:00:17.698050 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:17.697623    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerStarted","Data":"72d825088a12e1862f3d3a4be55fc8d359972dadf27dec557dc72d7387f6badb"}
Oct 30 21:00:17.700720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:17.700684    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"]
Oct 30 21:00:17.701145 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:17.701117    4852 generic.go:356] "Generic (PLEG): container finished" podID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerID="1d0b5c8ac393d03bc57d69938c71a12caa211e0cdeb8816a5233a09c58eaa996" exitCode=60
Oct 30 21:00:17.701480 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:17.701410    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerDied","Data":"1d0b5c8ac393d03bc57d69938c71a12caa211e0cdeb8816a5233a09c58eaa996"}
Oct 30 21:00:17.701571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:17.701482    4852 scope.go:122] "RemoveContainer" containerID="453fe2b7080d4d6e93c7bf4da6d6da7318adea008d9cdcf5bd0671aa420f155a"
Oct 30 21:00:17.707193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:17.706360    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" event={"ID":"bf0c9ee2-e911-4580-abc1-acdb42356ecc","Type":"ContainerStarted","Data":"50809c46ed3ff8adcc4d04efd534a998f1e0dd1f84c29ade5750850f5d490816"}
Oct 30 21:00:17.877667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:17.877636    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:18.023420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.023376    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d082bc30-f100-4dfd-99f8-476dbb765fed-kube-api-access-zjgk9\" (UniqueName: \"kubernetes.io/projected/d082bc30-f100-4dfd-99f8-476dbb765fed-kube-api-access-zjgk9\") pod \"d082bc30-f100-4dfd-99f8-476dbb765fed\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") "
Oct 30 21:00:18.023905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.023535    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d082bc30-f100-4dfd-99f8-476dbb765fed-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d082bc30-f100-4dfd-99f8-476dbb765fed-maas-api-service-ca\") pod \"d082bc30-f100-4dfd-99f8-476dbb765fed\" (UID: \"d082bc30-f100-4dfd-99f8-476dbb765fed\") "
Oct 30 21:00:18.023975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.023911    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d082bc30-f100-4dfd-99f8-476dbb765fed-maas-api-service-ca" pod "d082bc30-f100-4dfd-99f8-476dbb765fed" (UID: "d082bc30-f100-4dfd-99f8-476dbb765fed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:00:18.024233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.024213    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d082bc30-f100-4dfd-99f8-476dbb765fed-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:18.025572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.025547    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d082bc30-f100-4dfd-99f8-476dbb765fed-kube-api-access-zjgk9" pod "d082bc30-f100-4dfd-99f8-476dbb765fed" (UID: "d082bc30-f100-4dfd-99f8-476dbb765fed"). InnerVolumeSpecName "kube-api-access-zjgk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:00:18.125714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.125674    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjgk9\" (UniqueName: \"kubernetes.io/projected/d082bc30-f100-4dfd-99f8-476dbb765fed-kube-api-access-zjgk9\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:18.234935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.234836    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:18.799408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.798576    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"]
Oct 30 21:00:18.808406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.805899    4852 generic.go:356] "Generic (PLEG): container finished" podID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerID="50809c46ed3ff8adcc4d04efd534a998f1e0dd1f84c29ade5750850f5d490816" exitCode=60
Oct 30 21:00:18.808406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.806194    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"]
Oct 30 21:00:18.808406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.806253    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" event={"ID":"bf0c9ee2-e911-4580-abc1-acdb42356ecc","Type":"ContainerDied","Data":"50809c46ed3ff8adcc4d04efd534a998f1e0dd1f84c29ade5750850f5d490816"}
Oct 30 21:00:18.808406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.806292    4852 scope.go:122] "RemoveContainer" containerID="7917932b06e80fccf5430e21c3ffcbced7eba65ba0fcc6450641dbf197daa2ad"
Oct 30 21:00:18.842513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.841970    4852 generic.go:356] "Generic (PLEG): container finished" podID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerID="72d825088a12e1862f3d3a4be55fc8d359972dadf27dec557dc72d7387f6badb" exitCode=60
Oct 30 21:00:18.842513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.842080    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerDied","Data":"72d825088a12e1862f3d3a4be55fc8d359972dadf27dec557dc72d7387f6badb"}
Oct 30 21:00:18.848348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.848314    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt" event={"ID":"d082bc30-f100-4dfd-99f8-476dbb765fed","Type":"ContainerDied","Data":"a224c835215484942f35958e797f734cee207ed1a2264b3f1ba85f552a4d04b4"}
Oct 30 21:00:18.848669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.848654    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"
Oct 30 21:00:18.910331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.910283    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"]
Oct 30 21:00:18.923686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.923647    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889900-d7rmt"]
Oct 30 21:00:18.931720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.931648    4852 scope.go:122] "RemoveContainer" containerID="6ed609945684b51a449b4c27fc5e189ced507caef9729d1affcce30a141d1180"
Oct 30 21:00:18.981304 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:18.977144    4852 scope.go:122] "RemoveContainer" containerID="1d0b5c8ac393d03bc57d69938c71a12caa211e0cdeb8816a5233a09c58eaa996"
Oct 30 21:00:19.191989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.190082    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.206104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.206068    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:19.211977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.211019    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf0c9ee2-e911-4580-abc1-acdb42356ecc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c9ee2-e911-4580-abc1-acdb42356ecc-maas-api-service-ca\") pod \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") "
Oct 30 21:00:19.211977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.211166    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf0c9ee2-e911-4580-abc1-acdb42356ecc-kube-api-access-dcrfb\" (UniqueName: \"kubernetes.io/projected/bf0c9ee2-e911-4580-abc1-acdb42356ecc-kube-api-access-dcrfb\") pod \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\" (UID: \"bf0c9ee2-e911-4580-abc1-acdb42356ecc\") "
Oct 30 21:00:19.212623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.212596    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0c9ee2-e911-4580-abc1-acdb42356ecc-maas-api-service-ca" pod "bf0c9ee2-e911-4580-abc1-acdb42356ecc" (UID: "bf0c9ee2-e911-4580-abc1-acdb42356ecc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:00:19.251441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.241344    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0c9ee2-e911-4580-abc1-acdb42356ecc-kube-api-access-dcrfb" pod "bf0c9ee2-e911-4580-abc1-acdb42356ecc" (UID: "bf0c9ee2-e911-4580-abc1-acdb42356ecc"). InnerVolumeSpecName "kube-api-access-dcrfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:00:19.321205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.321143    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcrfb\" (UniqueName: \"kubernetes.io/projected/bf0c9ee2-e911-4580-abc1-acdb42356ecc-kube-api-access-dcrfb\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:19.321533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.321497    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0c9ee2-e911-4580-abc1-acdb42356ecc-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:19.355262 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.352059    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:19.426151 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.426043    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6af05da-c933-443a-a0a2-3877af1dc2ff-kube-api-access-md6p4\" (UniqueName: \"kubernetes.io/projected/e6af05da-c933-443a-a0a2-3877af1dc2ff-kube-api-access-md6p4\") pod \"e6af05da-c933-443a-a0a2-3877af1dc2ff\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") "
Oct 30 21:00:19.426339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.426200    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6af05da-c933-443a-a0a2-3877af1dc2ff-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6af05da-c933-443a-a0a2-3877af1dc2ff-maas-api-service-ca\") pod \"e6af05da-c933-443a-a0a2-3877af1dc2ff\" (UID: \"e6af05da-c933-443a-a0a2-3877af1dc2ff\") "
Oct 30 21:00:19.427204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.427170    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6af05da-c933-443a-a0a2-3877af1dc2ff-maas-api-service-ca" pod "e6af05da-c933-443a-a0a2-3877af1dc2ff" (UID: "e6af05da-c933-443a-a0a2-3877af1dc2ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:00:19.433644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.433608    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6af05da-c933-443a-a0a2-3877af1dc2ff-kube-api-access-md6p4" pod "e6af05da-c933-443a-a0a2-3877af1dc2ff" (UID: "e6af05da-c933-443a-a0a2-3877af1dc2ff"). InnerVolumeSpecName "kube-api-access-md6p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:00:19.542417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.534270    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md6p4\" (UniqueName: \"kubernetes.io/projected/e6af05da-c933-443a-a0a2-3877af1dc2ff-kube-api-access-md6p4\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:19.542417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.534306    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6af05da-c933-443a-a0a2-3877af1dc2ff-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:19.592420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.580866    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.702470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.699365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:19.912409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:19.854017    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:20.382842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.382799    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" path="/var/lib/kubelet/pods/d082bc30-f100-4dfd-99f8-476dbb765fed/volumes"
Oct 30 21:00:20.443341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.443227    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"
Oct 30 21:00:20.478411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.451959    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5" event={"ID":"e6af05da-c933-443a-a0a2-3877af1dc2ff","Type":"ContainerDied","Data":"beaa307f28009205be10fae3a1c5f5efc2dd913bbab16cc44afd1d334dd14530"}
Oct 30 21:00:20.478411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.452043    4852 scope.go:122] "RemoveContainer" containerID="72d825088a12e1862f3d3a4be55fc8d359972dadf27dec557dc72d7387f6badb"
Oct 30 21:00:20.521203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.521168    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq" event={"ID":"bf0c9ee2-e911-4580-abc1-acdb42356ecc","Type":"ContainerDied","Data":"a18248dab2362a0f7f71f22ca580b496f2b54d5ba7d1f01b372c52e1ef185861"}
Oct 30 21:00:20.521629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.521604    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"
Oct 30 21:00:20.559409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.558529    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:20.673869 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.672484    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:20.814015 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.809559    4852 scope.go:122] "RemoveContainer" containerID="50809c46ed3ff8adcc4d04efd534a998f1e0dd1f84c29ade5750850f5d490816"
Oct 30 21:00:20.853942 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.853901    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:00:20.892107 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:20.892014    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:21.060562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:21.060472    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"]
Oct 30 21:00:21.134353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:21.134309    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-b46fb7-29889900-xnqd5"]
Oct 30 21:00:21.140875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:21.140841    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"]
Oct 30 21:00:21.147795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:21.147758    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889900-kk2jq"]
Oct 30 21:00:21.558438 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:21.558375    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:00:22.005923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:22.005895    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" path="/var/lib/kubelet/pods/bf0c9ee2-e911-4580-abc1-acdb42356ecc/volumes"
Oct 30 21:00:22.006333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:22.006320    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" path="/var/lib/kubelet/pods/e6af05da-c933-443a-a0a2-3877af1dc2ff/volumes"
Oct 30 21:00:22.249516 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:22.249482    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:23.132494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:23.132455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:23.236587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:23.236539    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"]
Oct 30 21:00:23.236803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:23.236782    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy" containerID="cri-o://10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97" gracePeriod=30
Oct 30 21:00:24.005165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:24.005132    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:24.898581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:24.898547    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:25.516287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:25.516254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:25.608940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:25.608895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:26.241454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:26.241410    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:27.426720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:27.426681    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:28.419307 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.419273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:28.534878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.534852    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 21:00:28.586067 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.586029    4852 generic.go:356] "Generic (PLEG): container finished" podID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerID="10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97" exitCode=0
Oct 30 21:00:28.586235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.586133    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" event={"ID":"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7","Type":"ContainerDied","Data":"10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97"}
Oct 30 21:00:28.586235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.586140    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"
Oct 30 21:00:28.586235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.586190    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt" event={"ID":"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7","Type":"ContainerDied","Data":"e4c8483e4bd3a7de4287da4593edf5f98b4934c230e51e0ac0d708159298f82b"}
Oct 30 21:00:28.586235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.586222    4852 scope.go:122] "RemoveContainer" containerID="10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97"
Oct 30 21:00:28.599080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.599055    4852 scope.go:122] "RemoveContainer" containerID="10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97"
Oct 30 21:00:28.599449 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:28.599428    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97\": container with ID starting with 10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97 not found: ID does not exist" containerID="10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97"
Oct 30 21:00:28.599567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.599455    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97"} err="failed to get container status \"10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97\": rpc error: code = NotFound desc = could not find container \"10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97\": container with ID starting with 10b4f62bb642f8ed3cff5f9258350ab99622a98440d537aa045c62aa56d89e97 not found: ID does not exist"
Oct 30 21:00:28.639748 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.639720    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-certs\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.639923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.639822    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istiod-ca-cert\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.639923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.639881    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-podinfo\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.639923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.639914    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-envoy\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.640094 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.639970    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-certs" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:00:28.640094 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640034    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-socket\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.640094 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640089    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-data\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.640264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640143    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-token\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-token\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.640264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640182    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-credential-socket\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.640264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640226    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-kube-api-access-47mb7\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-kube-api-access-47mb7\") pod \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\" (UID: \"1f503e52-4e3f-42c4-a3bf-ed99f439a3a7\") "
Oct 30 21:00:28.640468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640260    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istiod-ca-cert" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:00:28.640531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640496    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-data" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:00:28.640644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640621    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-credential-socket" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:00:28.640724 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640669    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-socket" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:00:28.640896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640879    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.640977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640898    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.640977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640911    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.640977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640920    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.640977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.640929    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.642556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.642533    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-token" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:00:28.642678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.642654    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-envoy" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:00:28.642730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.642661    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-podinfo" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:00:28.642836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.642817    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-kube-api-access-47mb7" pod "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" (UID: "1f503e52-4e3f-42c4-a3bf-ed99f439a3a7"). InnerVolumeSpecName "kube-api-access-47mb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:00:28.742241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.742205    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.742433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.742251    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47mb7\" (UniqueName: \"kubernetes.io/projected/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-kube-api-access-47mb7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.742433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.742264    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.742433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.742277    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:00:28.913049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.913008    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"]
Oct 30 21:00:28.917932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:28.917904    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-7pzkt"]
Oct 30 21:00:29.421322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:29.421274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:30.005571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:30.005540    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" path="/var/lib/kubelet/pods/1f503e52-4e3f-42c4-a3bf-ed99f439a3a7/volumes"
Oct 30 21:00:30.219352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:30.219320    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:31.300498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:31.300441    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:33.213413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.212447    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.247410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.247342    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:00:33.318410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.317904    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.439821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.439775    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.444533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.444476    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.444793 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.444476    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.453423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.453359    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.499986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.499895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:33.518572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.518519    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.684555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.684507    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:33.820350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:33.820286    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.237465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.237241    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.268414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.238015    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:00:34.268414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.262584    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.313250 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.313190    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.367898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.333374    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.369878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.346537    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:00:34.370148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.370128    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:00:34.370297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.370284    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:00:34.440435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.435935    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.510408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.507459    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.510408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.507894    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:00:34.535408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.534872    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:00:34.535408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.535241    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:00:34.536059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.534805    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.698989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.694053    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:00:34.780604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.780515    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:00:34.790425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.783864    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:00:34.827808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.821725    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:00:34.827808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.822336    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:34.827808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.823897    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:00:34.827808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.824255    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:00:34.842874 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.842847    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:00:34.909476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:34.909419    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:36.322400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:36.322359    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:38.048131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:38.048093    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:39.988583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:39.988554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:41.517062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:41.517024    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:42.390023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:42.389986    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:42.482217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:42.482180    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:42.581301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:42.581267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:42.702188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:42.699503    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:42.849482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:42.849445    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:43.145567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:43.145527    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:43.389851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:43.389819    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:44.105411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.105277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:44.157970 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.157932    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"]
Oct 30 21:00:44.159086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159039    4852 cpu_manager.go:414] "containerMap: removing container" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy"
Oct 30 21:00:44.159258 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159239    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy"
Oct 30 21:00:44.159422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159374    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.159537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159522    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.159629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159618    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.159715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159703    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.159801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159790    4852 cpu_manager.go:414] "containerMap: removing container" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.159882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159871    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.159969 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.159959    4852 cpu_manager.go:414] "containerMap: removing container" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.160049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160039    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.160154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160144    4852 cpu_manager.go:414] "containerMap: removing container" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.160234 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160223    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.160318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160308    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.160426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160413    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.160537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160526    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.160620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160609    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.160718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160707    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.160798 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160786    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.160888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160878    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.160971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.160960    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.161205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161190    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.161315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161303    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.161435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161423    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.161551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161538    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f503e52-4e3f-42c4-a3bf-ed99f439a3a7" containerName="istio-proxy"
Oct 30 21:00:44.161645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161634    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.161723 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161713    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6af05da-c933-443a-a0a2-3877af1dc2ff" containerName="cleanup"
Oct 30 21:00:44.161825 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161812    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.161980 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.161968    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:44.162067 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.162056    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d082bc30-f100-4dfd-99f8-476dbb765fed" containerName="cleanup"
Oct 30 21:00:44.169688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.169663    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.274233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255273    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.274233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255343    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.274233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255403    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2aa6872-a8ec-463e-8524-040ce926883d-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.274233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255436    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2aa6872-a8ec-463e-8524-040ce926883d-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.274233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255555    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.274233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255670    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.274233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255739    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnjnp\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-kube-api-access-xnjnp\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.285427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.255769    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.285427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.275666    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.289619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.289556    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"]
Oct 30 21:00:44.402458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.401908    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424200    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-certs\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424318    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnjnp\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-kube-api-access-xnjnp\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424352    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424417    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424492    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424531    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424577    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2aa6872-a8ec-463e-8524-040ce926883d-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424605    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2aa6872-a8ec-463e-8524-040ce926883d-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.424776    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.425020    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.425260    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-data\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.425545    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-credential-socket\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.443467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.425991    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2aa6872-a8ec-463e-8524-040ce926883d-istiod-ca-cert\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.454419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.448771    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-envoy\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.454419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.450873    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-istio-token\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.487125 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.487050    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2aa6872-a8ec-463e-8524-040ce926883d-istio-podinfo\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.570411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.569123    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnjnp\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-kube-api-access-xnjnp\") pod \"e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") " pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:44.890435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:44.885985    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:45.426537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.426484    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:45.444534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.444482    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:45.446898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.445820    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:00:45.462129 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.461855    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:45.535410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.534490    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:45.659412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.658497    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:00:45.659412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.658988    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Oct 30 21:00:45.705585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.705533    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:45.725404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.725293    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:45.938686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.756605    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/health\": context deadline exceeded"
Oct 30 21:00:45.938686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.793456    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 21:00:45.938686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.793505    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:45.938686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.841459    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:00:45.938686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.911807    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:45.938686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:45.912164    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:00:46.427772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.427724    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 21:00:46.428867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.428844    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:00:46.471586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.471542    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.473110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.473074    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.477514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.477492    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:00:46.477700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.477686    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:00:46.552620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.552091    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.634854 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:00:46.634802    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2aa6872_a8ec_463e_8524_040ce926883d.slice/crio-cf3b5bc3eb8090790baacfd37cb7cd5a5c5b94c78ac4da442d9614a9480f16d8 WatchSource:0}: Error finding container cf3b5bc3eb8090790baacfd37cb7cd5a5c5b94c78ac4da442d9614a9480f16d8: Status 404 returned error can't find the container with id cf3b5bc3eb8090790baacfd37cb7cd5a5c5b94c78ac4da442d9614a9480f16d8
Oct 30 21:00:46.659135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.659077    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.719023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.715466    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.779175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.769188    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:00:46.779175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.769296    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:00:46.779175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.769337    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:00:46.779175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.769359    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"]
Oct 30 21:00:46.881414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.813094    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:00:46.881414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.813189    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:00:46.881414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.813239    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:00:46.881414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.862280    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.907126 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.905273    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.907126 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.905534    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 21:00:46.930408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.907490    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:00:46.930769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.930739    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:00:46.930973 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.930957    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:00:46.931623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.921539    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.957972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.921581    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:00:46.958519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.941294    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:00:46.962544 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:46.962509    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:00:47.504579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.504525    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 21:00:47.614411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.582707    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:00:47.626555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.625247    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:00:47.708410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.706666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:47.790639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.790577    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:00:47.791712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.791683    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:00:47.853450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.840886    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:00:47.853450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.851247    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:00:47.878428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.877545    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:00:47.929910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.909662    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:00:47.929910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.909946    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 21:00:47.929910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.914680    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:00:47.953257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:47.952362    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" event={"ID":"c2aa6872-a8ec-463e-8524-040ce926883d","Type":"ContainerStarted","Data":"cf3b5bc3eb8090790baacfd37cb7cd5a5c5b94c78ac4da442d9614a9480f16d8"}
Oct 30 21:00:48.063168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:48.042782    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:00:48.063168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:48.053108    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:00:48.085691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:48.080272    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:00:48.085691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:48.080329    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:00:48.118405 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:48.112832    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:00:48.985059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:48.985024    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" event={"ID":"c2aa6872-a8ec-463e-8524-040ce926883d","Type":"ContainerStarted","Data":"481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3"}
Oct 30 21:00:49.009243 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:49.009188    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podStartSLOduration=5.009175614 podStartE2EDuration="5.009175614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:49.006302214 +0000 UTC m=+1357.121036404" watchObservedRunningTime="2026-10-30 21:00:49.009175614 +0000 UTC m=+1357.123909803"
Oct 30 21:00:49.054626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:49.054504    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:49.889182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:49.889145    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:49.891595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:49.891562    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:00:49.891869 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:49.891837    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:00:50.094976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:50.094945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:50.576956 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:50.576926    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:50.678945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:50.678907    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:50.892957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:50.892850    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:00:50.892957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:50.892931    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:00:50.916816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:50.916757    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:51.053293 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:51.053256    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:51.583258 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:00:51.583209    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:00:51.889128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:51.889028    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:00:51.889128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:51.889109    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:00:52.759799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:52.759763    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:52.889052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:52.889004    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:00:52.889266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:52.889064    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:00:53.888676 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:53.888645    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:00:53.889164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:53.889135    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:00:54.270866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:54.270833    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:54.889689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:54.889655    4852 patch_prober.go:29] interesting pod/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:00:54.890199 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:54.889704    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.123:15021/healthz/ready\": dial tcp 10.133.0.123:15021: connect: connection refused"
Oct 30 21:00:55.055218 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.055179    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"]
Oct 30 21:00:55.056244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.056216    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0c9ee2-e911-4580-abc1-acdb42356ecc" containerName="cleanup"
Oct 30 21:00:55.060777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.060736    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.091307 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.091272    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"]
Oct 30 21:00:55.209953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.209858    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.209953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.209921    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.210246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.210220    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.210321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.210264    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.210447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.210430    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8118f70-aa12-47e3-8c81-03f576085993-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.210523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.210463    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.210523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.210499    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.210648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.210615    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stv8f\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-kube-api-access-stv8f\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.210690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.210668    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8118f70-aa12-47e3-8c81-03f576085993-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.312846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.312800    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.312860    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.312942    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8118f70-aa12-47e3-8c81-03f576085993-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.312971    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313010    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313059    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stv8f\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-kube-api-access-stv8f\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313090    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8118f70-aa12-47e3-8c81-03f576085993-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313118    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313187    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313346    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-credential-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313465    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-socket\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313797 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313768    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-data\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313793    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8118f70-aa12-47e3-8c81-03f576085993-istiod-ca-cert\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.313878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.313820    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-certs\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.315937 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.315910    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8118f70-aa12-47e3-8c81-03f576085993-istio-podinfo\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.316035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.315966    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-envoy\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.321666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.321642    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-istio-token\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.322312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.322283    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stv8f\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-kube-api-access-stv8f\") pod \"e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") " pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.417989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.417950    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:55.424983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.424958    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:55.746474 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.746446    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"]
Oct 30 21:00:55.747342 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:00:55.747314    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8118f70_aa12_47e3_8c81_03f576085993.slice/crio-9c3b0fd741b24fdf1cbccfa1072254e4848f09dbafc62331236276ba25e48eaf WatchSource:0}: Error finding container 9c3b0fd741b24fdf1cbccfa1072254e4848f09dbafc62331236276ba25e48eaf: Status 404 returned error can't find the container with id 9c3b0fd741b24fdf1cbccfa1072254e4848f09dbafc62331236276ba25e48eaf
Oct 30 21:00:55.750146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.750108    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:00:55.750238 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.750197    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:00:55.750277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.750242    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:00:55.893187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.893158    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:55.893683 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.893565    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:55.894471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:55.894453    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:00:56.052277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:56.052188    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" event={"ID":"b8118f70-aa12-47e3-8c81-03f576085993","Type":"ContainerStarted","Data":"7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f"}
Oct 30 21:00:56.052277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:56.052248    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" event={"ID":"b8118f70-aa12-47e3-8c81-03f576085993","Type":"ContainerStarted","Data":"9c3b0fd741b24fdf1cbccfa1072254e4848f09dbafc62331236276ba25e48eaf"}
Oct 30 21:00:56.076474 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:56.076422    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podStartSLOduration=1.076407185 podStartE2EDuration="1.076407185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:00:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:00:56.07218366 +0000 UTC m=+1364.186917849" watchObservedRunningTime="2026-10-30 21:00:56.076407185 +0000 UTC m=+1364.191141375"
Oct 30 21:00:56.425950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:56.425854    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:00:56.427308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:56.427280    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 21:00:56.427477 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:56.427327    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 21:00:56.476727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:56.476674    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:57.425866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:57.425831    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 21:00:57.426357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:57.425894    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 21:00:57.475562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:57.475522    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:58.426267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:58.426223    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 21:00:58.426783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:58.426284    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 21:00:58.475303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:58.475273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:00:59.425771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:59.425737    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 21:00:59.425954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:59.425794    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 21:00:59.976323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:00:59.976286    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:00.426182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:00.425971    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 21:01:00.426182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:00.426033    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 21:01:01.425648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:01.425614    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 21:01:01.426062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:01.425667    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 21:01:01.566078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:01.565939    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:02.425511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:02.425469    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 21:01:02.425700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:02.425540    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 21:01:03.511427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:03.511365    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:01:03.511905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:03.511812    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:01:03.512927 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:03.512905    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:01:03.943927 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:03.943848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:04.399804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:04.399763    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:06.637586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:06.637547    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:06.812812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:06.812772    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:07.434044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:07.433997    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:08.315609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:08.315570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:09.295241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:09.295206    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:09.961318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:09.961273    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"]
Oct 30 21:01:09.961894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:09.961547    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g" podUID="44f9c688-40a7-45f0-b51b-7691f9f3b07c" containerName="maas-api" containerID="cri-o://a9519ef9616567bde06ec3032c0fc2d7377e8151ae9dd9a21173e1899f3f3834" gracePeriod=30
Oct 30 21:01:10.005506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.005476    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.106110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.106074    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.195540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.195502    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.200871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.200840    4852 generic.go:356] "Generic (PLEG): container finished" podID="44f9c688-40a7-45f0-b51b-7691f9f3b07c" containerID="a9519ef9616567bde06ec3032c0fc2d7377e8151ae9dd9a21173e1899f3f3834" exitCode=0
Oct 30 21:01:10.201103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.201082    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g" event={"ID":"44f9c688-40a7-45f0-b51b-7691f9f3b07c","Type":"ContainerDied","Data":"a9519ef9616567bde06ec3032c0fc2d7377e8151ae9dd9a21173e1899f3f3834"}
Oct 30 21:01:10.255458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.255372    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 21:01:10.295602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.295550    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.377846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.377814    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44f9c688-40a7-45f0-b51b-7691f9f3b07c-kube-api-access-89lqt\" (UniqueName: \"kubernetes.io/projected/44f9c688-40a7-45f0-b51b-7691f9f3b07c-kube-api-access-89lqt\") pod \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") "
Oct 30 21:01:10.378009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.377946    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44f9c688-40a7-45f0-b51b-7691f9f3b07c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f9c688-40a7-45f0-b51b-7691f9f3b07c-openshift-service-ca\") pod \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") "
Oct 30 21:01:10.378009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.378002    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-metrics-certs\") pod \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") "
Oct 30 21:01:10.378159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.378139    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls\") pod \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\" (UID: \"44f9c688-40a7-45f0-b51b-7691f9f3b07c\") "
Oct 30 21:01:10.379674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.379639    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44f9c688-40a7-45f0-b51b-7691f9f3b07c-openshift-service-ca" pod "44f9c688-40a7-45f0-b51b-7691f9f3b07c" (UID: "44f9c688-40a7-45f0-b51b-7691f9f3b07c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:01:10.389434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.381750    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-metrics-certs" pod "44f9c688-40a7-45f0-b51b-7691f9f3b07c" (UID: "44f9c688-40a7-45f0-b51b-7691f9f3b07c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:01:10.389434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.382291    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f9c688-40a7-45f0-b51b-7691f9f3b07c-kube-api-access-89lqt" pod "44f9c688-40a7-45f0-b51b-7691f9f3b07c" (UID: "44f9c688-40a7-45f0-b51b-7691f9f3b07c"). InnerVolumeSpecName "kube-api-access-89lqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:01:10.389434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.383274    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls" pod "44f9c688-40a7-45f0-b51b-7691f9f3b07c" (UID: "44f9c688-40a7-45f0-b51b-7691f9f3b07c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:01:10.480619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.480536    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89lqt\" (UniqueName: \"kubernetes.io/projected/44f9c688-40a7-45f0-b51b-7691f9f3b07c-kube-api-access-89lqt\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:01:10.480619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.480567    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f9c688-40a7-45f0-b51b-7691f9f3b07c-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:01:10.480619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.480578    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:01:10.480619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.480588    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/44f9c688-40a7-45f0-b51b-7691f9f3b07c-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:01:10.560835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.560799    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:10.637371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:10.637330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:11.051463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:11.051377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:11.208985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:11.208941    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"
Oct 30 21:01:11.208985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:11.208963    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g" event={"ID":"44f9c688-40a7-45f0-b51b-7691f9f3b07c","Type":"ContainerDied","Data":"b555edd50d617cc3df7d8909dd5c91cf1f4272dd4d053d35d68cb29a2225b914"}
Oct 30 21:01:11.209227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:11.209033    4852 scope.go:122] "RemoveContainer" containerID="a9519ef9616567bde06ec3032c0fc2d7377e8151ae9dd9a21173e1899f3f3834"
Oct 30 21:01:11.235500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:11.235462    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"]
Oct 30 21:01:11.239873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:11.239846    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-b80638-74c57f4ff8-lxf5g"]
Oct 30 21:01:11.528052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:11.528021    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:12.006328 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:12.006301    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44f9c688-40a7-45f0-b51b-7691f9f3b07c" path="/var/lib/kubelet/pods/44f9c688-40a7-45f0-b51b-7691f9f3b07c/volumes"
Oct 30 21:01:12.397729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:12.397646    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:13.331630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:13.331580    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:14.329319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:14.329282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:15.136667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:15.136629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:15.457623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:15.457536    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:15.931292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:15.931259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:16.721741 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:16.721695    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:17.624345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:17.624306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:17.985466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:17.985356    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:18.083984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:18.083948    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:18.329433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:18.318108    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:19.588414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:19.583330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:19.842598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:19.842339    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:20.048696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.048410    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": context deadline exceeded"
Oct 30 21:01:20.085685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.085122    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:01:20.454418 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.415299    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.488413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.487582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:20.507574 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.507525    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.577409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.562013    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.663446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.631484    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.808561 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.807481    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.883187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.883144    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.901945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.893195    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.932540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.932491    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:20.933189 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:20.933153    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 21:01:21.035100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.035044    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.085871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.085763    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.086697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.086667    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:01:21.527497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.525014    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.527497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.525926    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:01:21.590573 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.590521    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:01:21.604410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.591438    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:01:21.643031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.642978    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.752411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.711551    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.781469 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.755583    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:01:21.827601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.807789    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.830871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.830845    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:01:21.942412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.903251    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.942412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.903529    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:01:21.942412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.903985    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:01:21.942412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.904909    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:01:21.942412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.905112    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:01:21.984412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.945592    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:21.986036 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.950492    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:01:21.987412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.987373    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:01:21.987612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:21.987596    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:01:22.061465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.048605    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:22.087410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.086520    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:01:22.307411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.283141    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:01:22.398602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.398512    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:22.565016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.564971    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:22.585909 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.585868    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:01:22.649625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.649521    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:22.829430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.825814    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:01:22.935247 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:01:22.879920    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:01:22.935247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.902696    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:22.954181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.952617    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:01:22.988519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.988477    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:01:22.997607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.997573    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:01:22.997910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.997882    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:22.999295 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:22.999267    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:01:23.022407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.019132    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:01:23.078404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.077650    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:01:23.094744 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.094709    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:01:23.095152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.095126    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:01:23.103636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.102131    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:01:23.127410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.103976    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:01:23.127410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.104592    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:01:23.127410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.105076    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:01:23.138325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:23.138276    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:01:24.251186 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:24.251153    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:25.335648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:25.335613    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:26.535499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:26.535455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:27.642913 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:27.642875    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:27.941207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:27.941131    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:28.049022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:28.048990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:28.743940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:28.743907    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.142845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:29.142812    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.248428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:29.248363    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.915159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:29.915124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:31.103356 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:31.103322    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:31.872496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:31.872467    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:32.730757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:32.730725    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:33.646056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:33.646014    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:34.632285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:34.632255    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:35.530453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:35.530371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:35.634550 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:35.634509    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:36.547951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:36.547908    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:37.310151 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:37.310102    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:38.117092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:38.117061    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:38.963212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:38.963176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:39.813728 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:39.813695    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:41.865063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:41.865004    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:41.966446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:41.965942    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.006433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:41.986274    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.008120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.008076    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.028872 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.028259    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.076681 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.076287    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:01:42.076681 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.076375    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.127995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.125242    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.145976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.131024    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.146903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.146602    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.266408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:41.874380    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:42.597025 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.596975    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.897179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.895871    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:42.897179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:42.896467    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:01:43.103975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:43.100346    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:01:43.530470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:43.530433    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:44.638558 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:44.638528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:45.640269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:45.640234    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:46.597683 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:46.597641    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:47.415073 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:47.415022    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:47.912678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:47.912647    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:48.235934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:48.235856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:49.168204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.168166    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:49.257930 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.257887    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:49.556755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.556710    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"]
Oct 30 21:01:49.557368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.557349    4852 cpu_manager.go:414] "containerMap: removing container" podUID="44f9c688-40a7-45f0-b51b-7691f9f3b07c" containerName="maas-api"
Oct 30 21:01:49.557368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.557369    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44f9c688-40a7-45f0-b51b-7691f9f3b07c" containerName="maas-api"
Oct 30 21:01:49.557595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.557480    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="44f9c688-40a7-45f0-b51b-7691f9f3b07c" containerName="maas-api"
Oct 30 21:01:49.562157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.562133    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.569512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.569491    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-fa48f3-openshift-default-dockercfg-r2ghv\""
Oct 30 21:01:49.575231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.575049    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"]
Oct 30 21:01:49.606526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.606493    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:49.650752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.650715    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-podinfo\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.651319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651294    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-socket\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.651577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651540    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-envoy\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.651711 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651607    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-token\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.651783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651723    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-certs\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.651849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651807    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lc4z\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-kube-api-access-6lc4z\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.651849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651844    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istiod-ca-cert\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.651960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651945    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-data\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.652011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.651964    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-credential-socket\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.737434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.737381    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:49.752832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.752798    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lc4z\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-kube-api-access-6lc4z\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.752845    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istiod-ca-cert\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.752904    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-data\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.752919    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-credential-socket\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753019    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-podinfo\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753044    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-socket\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753080    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-envoy\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753131    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-token\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753187    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-certs\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753527 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753352    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-data\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753527 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753442    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-socket\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753527 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753513    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-credential-socket\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753556    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-certs\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.753784 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.753763    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istiod-ca-cert\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.755563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.755539    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-envoy\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.756147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.756125    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-podinfo\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.763741 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.763711    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lc4z\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-kube-api-access-6lc4z\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.764057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.764039    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-token\") pod \"e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") " pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:49.881036 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:49.880931    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:50.184282 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.184255    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"]
Oct 30 21:01:50.185529 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:01:50.185495    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934f053b_f20d_44b6_a98e_dbb8f2c767f8.slice/crio-4128ab4dcfa7c37eb2bfa959a1ba3b031342657a7609cc504348c390f10ef6af WatchSource:0}: Error finding container 4128ab4dcfa7c37eb2bfa959a1ba3b031342657a7609cc504348c390f10ef6af: Status 404 returned error can't find the container with id 4128ab4dcfa7c37eb2bfa959a1ba3b031342657a7609cc504348c390f10ef6af
Oct 30 21:01:50.188299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.188255    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:01:50.188445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.188345    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:01:50.188493 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.188459    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:01:50.190122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.190096    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:50.392461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.392420    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" event={"ID":"934f053b-f20d-44b6-a98e-dbb8f2c767f8","Type":"ContainerStarted","Data":"c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8"}
Oct 30 21:01:50.392667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.392521    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" event={"ID":"934f053b-f20d-44b6-a98e-dbb8f2c767f8","Type":"ContainerStarted","Data":"4128ab4dcfa7c37eb2bfa959a1ba3b031342657a7609cc504348c390f10ef6af"}
Oct 30 21:01:50.412778 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.412722    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podStartSLOduration=1.412707937 podStartE2EDuration="1.412707937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:50.411262015 +0000 UTC m=+1418.525996218" watchObservedRunningTime="2026-10-30 21:01:50.412707937 +0000 UTC m=+1418.527442124"
Oct 30 21:01:50.882093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:50.882056    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:51.136444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:51.136353    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:51.557405 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:01:51.557361    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:01:51.887360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:51.887282    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:52.185718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:52.185633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:52.406156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:52.406127    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:52.407087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:52.407065    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:01:52.757028 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:52.757000    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:52.867529 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:52.867500    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:53.325736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:53.325702    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:53.723743 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:53.723660    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:53.974954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:53.974877    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.185131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:54.185096    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.372409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:54.366216    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.489413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:54.486644    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:56.958412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:56.957478    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 21:01:56.958969 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:56.958737    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:56.992412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:56.991516    4852 patch_prober.go:29] interesting pod/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:01:56.992412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:56.991585    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:57.023411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.022578    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:57.023411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.023032    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:01:57.110410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.105506    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:57.110410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.105704    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:57.110410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.106107    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:57.110410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.108236    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:01:57.158412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.148487    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:57.636413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.636013    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:57.706555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:57.706503    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:01:58.124488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:58.124450    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:59.120432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:59.115356    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:59.221095 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:59.221058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:59.957099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:01:59.957064    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:00.469530 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.469485    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:00.545570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.545538    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"]
Oct 30 21:02:00.550507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.550484    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.555926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.555866    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-176db5-dockercfg-c5n7s\""
Oct 30 21:02:00.555926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.555876    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-176db5\""
Oct 30 21:02:00.559537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.559505    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"]
Oct 30 21:02:00.569967 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.569919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:00.592619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.592590    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"]
Oct 30 21:02:00.597358 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.597334    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:00.607743 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.607125    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wpzw\" (UniqueName: \"kubernetes.io/projected/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-kube-api-access-9wpzw\") pod \"payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.607743 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.607277    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.610279 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.610240    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"]
Oct 30 21:02:00.722500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.712083    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.722500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.712163    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzh9\" (UniqueName: \"kubernetes.io/projected/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-kube-api-access-6vzh9\") pod \"payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:00.722500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.712200    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:00.722500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.712355    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wpzw\" (UniqueName: \"kubernetes.io/projected/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-kube-api-access-9wpzw\") pod \"payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.722500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.713685    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.742443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.741988    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wpzw\" (UniqueName: \"kubernetes.io/projected/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-kube-api-access-9wpzw\") pod \"payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.813971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.813933    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzh9\" (UniqueName: \"kubernetes.io/projected/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-kube-api-access-6vzh9\") pod \"payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:00.814186 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.814089    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:00.815006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.814978    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:00.823801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.823773    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzh9\" (UniqueName: \"kubernetes.io/projected/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-kube-api-access-6vzh9\") pod \"payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:00.867702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.867647    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:00.915462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:00.915426    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:01.158950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.158923    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"]
Oct 30 21:02:01.159150 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:02:01.159132    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bb5b9c_3a30_416c_ab6d_8f9f8483c73b.slice/crio-be9781bcb1e768244574b1c25bf1294aa874b6828d75e5c745f43ccae7e68ccd WatchSource:0}: Error finding container be9781bcb1e768244574b1c25bf1294aa874b6828d75e5c745f43ccae7e68ccd: Status 404 returned error can't find the container with id be9781bcb1e768244574b1c25bf1294aa874b6828d75e5c745f43ccae7e68ccd
Oct 30 21:02:01.219088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.219053    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"]
Oct 30 21:02:01.219218 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:02:01.219156    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7e6386_96d1_42cc_b5d0_a59ed2f2dfff.slice/crio-9d9eeb5f68ec25a3d82f17d34b52f8e6cdbffa694f3f53082d908d6788092a68 WatchSource:0}: Error finding container 9d9eeb5f68ec25a3d82f17d34b52f8e6cdbffa694f3f53082d908d6788092a68: Status 404 returned error can't find the container with id 9d9eeb5f68ec25a3d82f17d34b52f8e6cdbffa694f3f53082d908d6788092a68
Oct 30 21:02:01.230974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.230943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:01.725122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.725020    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" event={"ID":"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff","Type":"ContainerStarted","Data":"735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d"}
Oct 30 21:02:01.725122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.725085    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" event={"ID":"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff","Type":"ContainerStarted","Data":"9d9eeb5f68ec25a3d82f17d34b52f8e6cdbffa694f3f53082d908d6788092a68"}
Oct 30 21:02:01.725688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.725135    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:01.727074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.727048    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" event={"ID":"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b","Type":"ContainerStarted","Data":"fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324"}
Oct 30 21:02:01.727210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.727090    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" event={"ID":"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b","Type":"ContainerStarted","Data":"be9781bcb1e768244574b1c25bf1294aa874b6828d75e5c745f43ccae7e68ccd"}
Oct 30 21:02:01.727210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.727130    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:01.760923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.760870    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" podStartSLOduration=1.760857454 podStartE2EDuration="1.760857454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:01.759288413 +0000 UTC m=+1429.874022602" watchObservedRunningTime="2026-10-30 21:02:01.760857454 +0000 UTC m=+1429.875591644"
Oct 30 21:02:01.779458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:01.779404    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" podStartSLOduration=1.77937651 podStartE2EDuration="1.77937651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:01.776883424 +0000 UTC m=+1429.891617613" watchObservedRunningTime="2026-10-30 21:02:01.77937651 +0000 UTC m=+1429.894110699"
Oct 30 21:02:02.236982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:02.236945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:03.023443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:03.023409    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:03.826178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:03.826145    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:04.026966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.026903    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:04.569830 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.569787    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"]
Oct 30 21:02:04.575980 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.575373    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.582510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.582484    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-fa48f3-serving-cert\""
Oct 30 21:02:04.604710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.588649    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"]
Oct 30 21:02:04.620595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.620547    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:04.671164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.671121    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.671164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.671163    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-metrics-certs\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.671420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.671340    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnb7l\" (UniqueName: \"kubernetes.io/projected/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-kube-api-access-wnb7l\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.671490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.671473    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-maas-api-tls\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.720351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.720322    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:04.776870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.773548    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnb7l\" (UniqueName: \"kubernetes.io/projected/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-kube-api-access-wnb7l\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.776870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.773699    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-maas-api-tls\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.776870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.773869    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.776870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.773894    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-metrics-certs\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.776870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.774909    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.782294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.782250    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-maas-api-tls\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.786508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.783903    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnb7l\" (UniqueName: \"kubernetes.io/projected/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-kube-api-access-wnb7l\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.786508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.785127    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-metrics-certs\") pod \"maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:04.820518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.820423    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:04.899852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:04.899821    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:05.185766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:05.185727    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"]
Oct 30 21:02:05.186250 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:02:05.186226    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd45ccd_a11a_455d_bde1_5f03ac8a7a26.slice/crio-80bc2db8824f27d951fe378e01ad913fde2b9f951a573c8e2913e203476adc3b WatchSource:0}: Error finding container 80bc2db8824f27d951fe378e01ad913fde2b9f951a573c8e2913e203476adc3b: Status 404 returned error can't find the container with id 80bc2db8824f27d951fe378e01ad913fde2b9f951a573c8e2913e203476adc3b
Oct 30 21:02:05.676590 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:05.676563    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:05.767274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:05.764472    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl" event={"ID":"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26","Type":"ContainerStarted","Data":"cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf"}
Oct 30 21:02:05.767274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:05.764542    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl" event={"ID":"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26","Type":"ContainerStarted","Data":"80bc2db8824f27d951fe378e01ad913fde2b9f951a573c8e2913e203476adc3b"}
Oct 30 21:02:05.767274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:05.766217    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:05.787470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:05.787381    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl" podStartSLOduration=1.337846691 podStartE2EDuration="1.787364606s" totalImagesPullingTime="449.517915ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:05.786099246 +0000 UTC m=+1433.900833437" watchObservedRunningTime="2026-10-30 21:02:05.787364606 +0000 UTC m=+1433.902098796"
Oct 30 21:02:06.640703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:06.640656    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:07.633175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:07.633134    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:08.426670 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:08.426615    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:08.533888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:08.533795    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:09.622032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:09.621995    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:10.523082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:10.523048    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:10.784774 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:10.784671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:11.223589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:11.223495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:11.454725 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:11.454691    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:11.554642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:11.554593    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:11.791420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:11.791377    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:02:12.310829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:12.310796    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:12.737716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:12.737635    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:02:12.737716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:12.737707    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:02:13.146643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:13.146576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:14.015877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:14.015842    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:14.822732 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:14.822696    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:15.353032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:15.352995    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:15.452610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:15.452576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:15.860548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:15.860513    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:16.695118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.695081    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:16.825251 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.825200    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:16.893154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.893113    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"]
Oct 30 21:02:16.898478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.898440    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:16.925719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.925688    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"]
Oct 30 21:02:16.940513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.940068    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-8fa15b\""
Oct 30 21:02:16.940513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.940125    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-8fa15b-dockercfg-w2c8n\""
Oct 30 21:02:16.995035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.994934    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbz2b\" (UniqueName: \"kubernetes.io/projected/6a24bc29-1110-4746-9bca-5be86231f077-kube-api-access-dbz2b\") pod \"payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:16.997357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:16.995449    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a24bc29-1110-4746-9bca-5be86231f077-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:17.134500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:17.119365    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbz2b\" (UniqueName: \"kubernetes.io/projected/6a24bc29-1110-4746-9bca-5be86231f077-kube-api-access-dbz2b\") pod \"payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:17.281675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:17.281630    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a24bc29-1110-4746-9bca-5be86231f077-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:17.282844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:17.282782    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a24bc29-1110-4746-9bca-5be86231f077-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:17.301217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:17.301173    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbz2b\" (UniqueName: \"kubernetes.io/projected/6a24bc29-1110-4746-9bca-5be86231f077-kube-api-access-dbz2b\") pod \"payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:17.521960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:17.521922    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:17.897035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:17.896943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:18.010042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.010000    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:02:18.085829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.075223    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:18.219466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.204576    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:18.219466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.204953    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:02:18.219466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.208575    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:18.219466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.209033    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:02:18.631506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.381592    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:18.667267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.661465    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:18.667267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.663096    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:18.716071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.716036    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"]
Oct 30 21:02:18.978861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:18.978340    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.049079 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.024500    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:02:19.049079 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.027153    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 21:02:19.057499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.053484    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:02:19.091422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.085974    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.091422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.086229    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:02:19.117313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.117270    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.228541 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.228498    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.228803    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.228838    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.228862    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.245510    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.245588    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.245627    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.245658    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:02:19.260053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.248644    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.358491 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.331717    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:19.429866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.429817    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" event={"ID":"6a24bc29-1110-4746-9bca-5be86231f077","Type":"ContainerStarted","Data":"9566d48216d7e16dac786fb4c60b359f0beb9cf1a33f8a0ac6a009a5f78d3162"}
Oct 30 21:02:19.671703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.671661    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.695916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.677810    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:02:19.708442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.696481    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:02:19.745254 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.695497    4852 patch_prober.go:29] interesting pod/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:02:19.745568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.745537    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:19.746036 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:19.746019    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:02:20.044414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.014268    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:20.073413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.031482    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 21:02:20.074503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.031522    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:20.128850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.102079    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:20.129478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.117512    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:20.151581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.117621    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 21:02:20.269329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.178530    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:02:20.269660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.269639    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:02:20.269854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.269838    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:02:20.301470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.268771    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:02:20.301470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.268733    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:20.301470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.268791    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:02:20.383995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.383912    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:02:20.411684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.396957    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:20.704442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.700928    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:02:20.729599 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.704692    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:02:20.741916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.741876    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:02:20.834458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.834425    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:02:20.836700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.836674    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:02:20.851974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.845138    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:02:20.851974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.845199    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:02:20.851974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.845221    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:02:20.851974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.845242    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:02:20.895529 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.895180    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:02:20.897514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.897130    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:02:20.904173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.904135    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:02:20.906607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:20.906556    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:02:21.015742 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:21.015574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:21.575943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:21.575901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:21.584105 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:02:21.584071    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:02:21.678447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:21.678376    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" event={"ID":"6a24bc29-1110-4746-9bca-5be86231f077","Type":"ContainerStarted","Data":"63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214"}
Oct 30 21:02:21.678626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:21.678489    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:21.695965 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:21.695909    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" podStartSLOduration=5.695896193 podStartE2EDuration="5.695896193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:21.693704056 +0000 UTC m=+1449.808438246" watchObservedRunningTime="2026-10-30 21:02:21.695896193 +0000 UTC m=+1449.810630386"
Oct 30 21:02:22.138010 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.137970    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"]
Oct 30 21:02:22.142716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.142691    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.149024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.149001    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-e53c37-openshift-default-dockercfg-kkgwd\""
Oct 30 21:02:22.189536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.189498    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"]
Oct 30 21:02:22.206100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206063    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-istio-token\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206139    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-credential-socket\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206245    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-socket\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206271    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-data\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206371    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-envoy\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206535 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206515    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f7322-e13b-41bc-b47b-39390953298e-istiod-ca-cert\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206557    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f7322-e13b-41bc-b47b-39390953298e-istio-podinfo\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206583    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfn8x\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-kube-api-access-dfn8x\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.206684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.206674    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-certs\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.210052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.210008    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:22.311469 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311426    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-istio-token\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311524    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-credential-socket\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311576    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-socket\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311617    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-data\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311663    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-envoy\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311723    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f7322-e13b-41bc-b47b-39390953298e-istiod-ca-cert\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311755    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f7322-e13b-41bc-b47b-39390953298e-istio-podinfo\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311783    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfn8x\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-kube-api-access-dfn8x\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.311871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.311844    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-certs\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.312374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.312349    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-certs\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.312969 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.312946    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-credential-socket\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.313205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.313184    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-socket\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.313466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.313445    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-data\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.333483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.333214    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f7322-e13b-41bc-b47b-39390953298e-istiod-ca-cert\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.338460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.338430    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-envoy\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.339816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.339784    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:22.348414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.347662    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f7322-e13b-41bc-b47b-39390953298e-istio-podinfo\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.366667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.362995    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-istio-token\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.367620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.367554    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfn8x\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-kube-api-access-dfn8x\") pod \"e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") " pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.487782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.487684    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:22.832124 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.832090    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"]
Oct 30 21:02:22.832587 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:02:22.832292    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4f7322_e13b_41bc_b47b_39390953298e.slice/crio-85b4c0d84227da70d97bd60b2ec2564bb2b181d732fb8adb0a0ed1f73cfcae0a WatchSource:0}: Error finding container 85b4c0d84227da70d97bd60b2ec2564bb2b181d732fb8adb0a0ed1f73cfcae0a: Status 404 returned error can't find the container with id 85b4c0d84227da70d97bd60b2ec2564bb2b181d732fb8adb0a0ed1f73cfcae0a
Oct 30 21:02:22.835490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.835453    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:22.835595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.835526    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:22.835595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:22.835561    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:23.597266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:23.597211    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:23.708256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:23.705473    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" event={"ID":"3c4f7322-e13b-41bc-b47b-39390953298e","Type":"ContainerStarted","Data":"55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52"}
Oct 30 21:02:23.708256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:23.705557    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" event={"ID":"3c4f7322-e13b-41bc-b47b-39390953298e","Type":"ContainerStarted","Data":"85b4c0d84227da70d97bd60b2ec2564bb2b181d732fb8adb0a0ed1f73cfcae0a"}
Oct 30 21:02:23.743758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:23.743708    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podStartSLOduration=1.7436943089999999 podStartE2EDuration="1.743694309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:23.738007846 +0000 UTC m=+1451.852742035" watchObservedRunningTime="2026-10-30 21:02:23.743694309 +0000 UTC m=+1451.858428566"
Oct 30 21:02:24.023856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.023822    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:24.450510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.450374    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:24.488936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.488893    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:24.495735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.495705    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:24.714875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.714779    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:24.715975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.715933    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:02:24.752053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.752006    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:24.947064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.946661    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"]
Oct 30 21:02:24.958302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.958269    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"]
Oct 30 21:02:24.958534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.958404    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.960211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.960178    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:24.966856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.966788    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-74f9f4-openshift-default-dockercfg-cf7zm\""
Oct 30 21:02:24.997822 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.997774    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-credential-socket\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.997864    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-podinfo\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.997894    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-socket\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.997987    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-envoy\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.998016    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-data\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.998118    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istiod-ca-cert\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.998142    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk6st\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-kube-api-access-qk6st\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.998211    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-token\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:24.998423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:24.998295    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-certs\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.070993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.070922    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:25.112597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.112545    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-certs\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.114600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.113401    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-credential-socket\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.114956 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.114908    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-podinfo\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.115110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.115094    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-socket\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.115417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.115357    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-envoy\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.117654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.115539    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-data\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.119827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.119251    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-credential-socket\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.120152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.120116    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istiod-ca-cert\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.121706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.121682    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk6st\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-kube-api-access-qk6st\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.130045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.130017    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-token\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.143143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.118202    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-data\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.148086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.122578    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-certs\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.151772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.122836    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-socket\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.153700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.121711    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istiod-ca-cert\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.202910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.202881    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk6st\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-kube-api-access-qk6st\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.223294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.222730    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-envoy\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.237510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.237429    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-token\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.238270 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.238215    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-podinfo\") pod \"e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") " pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.281094 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.280808    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:25.591917 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:02:25.591873    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3bf68a_44ac_4f67_a4f6_8e79c517bac1.slice/crio-85cec5e0e43a39fb6efca6ecaca1d94d8f63ab5bd857e987715557aeeb697a82 WatchSource:0}: Error finding container 85cec5e0e43a39fb6efca6ecaca1d94d8f63ab5bd857e987715557aeeb697a82: Status 404 returned error can't find the container with id 85cec5e0e43a39fb6efca6ecaca1d94d8f63ab5bd857e987715557aeeb697a82
Oct 30 21:02:25.592616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.592476    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"]
Oct 30 21:02:25.594926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.594894    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:25.595028 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.594972    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:25.595028 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.595006    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:25.733325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.733221    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" event={"ID":"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1","Type":"ContainerStarted","Data":"58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182"}
Oct 30 21:02:25.733325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.733280    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" event={"ID":"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1","Type":"ContainerStarted","Data":"85cec5e0e43a39fb6efca6ecaca1d94d8f63ab5bd857e987715557aeeb697a82"}
Oct 30 21:02:25.762498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:25.762433    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podStartSLOduration=1.7624135 podStartE2EDuration="1.7624135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:25.757434482 +0000 UTC m=+1453.872168672" watchObservedRunningTime="2026-10-30 21:02:25.7624135 +0000 UTC m=+1453.877147691"
Oct 30 21:02:26.181887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:26.181824    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:26.286793 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:26.281772    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:26.286793 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:26.282306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:26.373691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:26.373603    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:26.739148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:26.739116    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:26.740121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:26.740098    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:02:27.312155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:27.312108    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:27.413859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:27.413803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:27.539434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:27.538576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:28.144122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.144092    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:28.242415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.242372    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:28.764327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.764286    4852 generic.go:356] "Generic (PLEG): container finished" podID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerID="c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8" exitCode=0
Oct 30 21:02:28.764766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.764367    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" event={"ID":"934f053b-f20d-44b6-a98e-dbb8f2c767f8","Type":"ContainerDied","Data":"c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8"}
Oct 30 21:02:28.765119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.765103    4852 scope.go:122] "RemoveContainer" containerID="c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8"
Oct 30 21:02:28.766709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.766684    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:28.766783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.766759    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:28.766826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:28.766794    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:29.046843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:29.046767    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:29.453014 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:29.452917    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:29.583008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:29.582965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:29.818247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:29.818208    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" event={"ID":"934f053b-f20d-44b6-a98e-dbb8f2c767f8","Type":"ContainerStarted","Data":"9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443"}
Oct 30 21:02:29.908514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:29.898734    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:02:29.908514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:29.907519    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 21:02:29.908514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:29.907578    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 21:02:30.230888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:30.230804    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:30.881983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:30.881951    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 21:02:30.882382 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:30.882002    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 21:02:31.210846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:31.210747    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:31.332408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:31.331735    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:31.896410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:31.890800    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 21:02:31.896410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:31.890864    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 21:02:32.274412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:32.270861    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:32.382412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:32.382341    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:32.481825 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:32.481786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:32.721476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:32.719618    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:02:32.883249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:32.883015    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 21:02:32.883249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:32.883077    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 21:02:33.225640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:33.225603    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:33.881883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:33.881844    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 21:02:33.882104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:33.881902    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 21:02:34.229783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:34.229698    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:34.882101 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:34.882064    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 21:02:34.882350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:34.882133    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 21:02:35.042400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:35.042338    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:35.303858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:35.303820    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:35.583496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:35.583377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:35.709987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:35.709950    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:35.886505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:35.886377    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:02:35.887025 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:35.886978    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:02:35.888132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:35.888106    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:02:36.137567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:36.137436    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:36.626830 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:36.626795    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:37.455596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:37.455559    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:38.159803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.159768    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"]
Oct 30 21:02:38.163786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.163759    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.169938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.169906    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-b46fb7-openshift-default-dockercfg-r9c4z\""
Oct 30 21:02:38.176096 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.176066    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"]
Oct 30 21:02:38.285264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285234    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-data\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285288    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-token\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285451    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-podinfo\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285509    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vwx5\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-kube-api-access-4vwx5\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285551    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-envoy\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285609    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-socket\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285649    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-credential-socket\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285702    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istiod-ca-cert\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.285930 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.285742    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-certs\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.301775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.301738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.387286    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-data\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.387354    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-token\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.387481    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-podinfo\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.387551    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vwx5\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-kube-api-access-4vwx5\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.387684    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-envoy\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.387903    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-socket\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.387951    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-credential-socket\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.388070    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istiod-ca-cert\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.388114    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-certs\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.388926    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-certs\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.391827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.389193    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-data\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.399777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.399747    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-podinfo\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.400108 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.400081    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-token\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.400598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.400572    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-credential-socket\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.401190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.401164    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istiod-ca-cert\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.401616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.401560    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-socket\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.412773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.411524    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-envoy\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.412773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.411933    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:38.419114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.419083    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vwx5\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-kube-api-access-4vwx5\") pod \"e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") " pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.480355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.480320    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:38.501611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.501581    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:38.771554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.771524    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"]
Oct 30 21:02:38.771694 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:02:38.771664    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7129cda6_8378_4d30_b0e4_7a7bcbcd6196.slice/crio-0ae840c635d3d5d7285369b5ee564bb5b56d86969d2e05e972504595b6bf53bc WatchSource:0}: Error finding container 0ae840c635d3d5d7285369b5ee564bb5b56d86969d2e05e972504595b6bf53bc: Status 404 returned error can't find the container with id 0ae840c635d3d5d7285369b5ee564bb5b56d86969d2e05e972504595b6bf53bc
Oct 30 21:02:38.774556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.774524    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:38.774656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.774589    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:38.774656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:38.774620    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:39.002867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:39.002830    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" event={"ID":"7129cda6-8378-4d30-b0e4-7a7bcbcd6196","Type":"ContainerStarted","Data":"b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137"}
Oct 30 21:02:39.003974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:39.003946    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" event={"ID":"7129cda6-8378-4d30-b0e4-7a7bcbcd6196","Type":"ContainerStarted","Data":"0ae840c635d3d5d7285369b5ee564bb5b56d86969d2e05e972504595b6bf53bc"}
Oct 30 21:02:39.031561 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:39.029966    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podStartSLOduration=1.029940327 podStartE2EDuration="1.029940327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:39.027705965 +0000 UTC m=+1467.142440155" watchObservedRunningTime="2026-10-30 21:02:39.029940327 +0000 UTC m=+1467.144674518"
Oct 30 21:02:39.319097 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:39.318989    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:39.481020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:39.480935    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:39.482604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:39.482571    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:39.482790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:39.482631    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:40.244518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:40.239887    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:40.348961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:40.348926    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:40.480841 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:40.480804    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:40.481605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:40.481567    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:41.214936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:41.214895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:41.481661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:41.481570    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:41.482114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:41.481655    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:41.972942 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:41.972901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:42.481769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:42.481719    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:42.482231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:42.481790    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:43.079861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:43.079782    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:43.493463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:43.480713    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:43.493463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:43.480781    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:43.669331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:43.669008    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:43.775414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:43.765938    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:44.037475 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.035920    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:44.142413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.142073    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.373476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.356912    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.404414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.403753    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.482932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.482886    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:44.483245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.483209    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:44.715472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.714493    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.748413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.733477    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.771413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.770518    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.789413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.782506    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.845412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.844525    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:02:44.849421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.845830    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.902412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.899054    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.902412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.900795    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:44.917412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:44.915631    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.091413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.084707    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/health\": context deadline exceeded"
Oct 30 21:02:45.145413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.144541    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.145413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.145312    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:02:45.361472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.357869    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:02:45.390412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.366556    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:02:45.521413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.506212    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:02:45.521413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.507693    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:02:45.590411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.589561    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:45.590411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.589627    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:45.617468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.616360    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.747934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.747888    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/health\": context deadline exceeded"
Oct 30 21:02:45.757407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.752700    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.757748 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.756777    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.834074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.756813    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:02:45.834501 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.834470    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:02:45.840341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.835818    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:02:45.858721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.784602    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.858721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.845112    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:02:45.858721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.846161    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.858721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.850829    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:02:45.858721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.850887    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:02:45.858721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.850925    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:02:45.908625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.908527    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.921410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.914474    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:45.946564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.946504    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:02:45.986413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:45.947127    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:02:46.043898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.015879    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:02:46.160607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.160491    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:46.334468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.334432    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:02:46.335063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.335037    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:02:46.352345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.352315    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:02:46.387720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.374413    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:46.511659 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.511609    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:02:46.605142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.605109    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:02:46.628403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.625003    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:02:46.644417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.642612    4852 patch_prober.go:29] interesting pod/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:02:46.644417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.642693    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.132:15021/healthz/ready\": dial tcp 10.133.0.132:15021: connect: connection refused"
Oct 30 21:02:46.651412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.650466    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:02:46.655985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.655961    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:02:46.656242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.656230    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:02:46.658848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.658825    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:02:46.658998 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.658871    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:02:46.668624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.662419    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:02:46.668624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.668542    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:02:46.676461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:46.673325    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:47.495906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:47.495876    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:47.496379 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:47.496202    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:47.497010 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:47.496991    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:02:48.273156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:48.273111    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:49.276159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:49.276124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:50.349703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:50.349671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:50.815982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:50.815945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:51.298255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:51.298223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:51.411003 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:51.410949    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:51.558492 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:02:51.558382    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:02:51.746894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:51.746851    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:51.836901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:51.836816    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:52.328378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:52.328348    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:53.129415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:53.129364    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:53.511111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:53.511075    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:53.611950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:53.611921    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:54.295897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:54.295858    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:55.158427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:55.158336    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:55.256612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:55.256580    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:55.439075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:55.438985    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:55.825936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:55.825901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:55.926353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:55.926308    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:56.275590 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.275548    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:56.286757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.286726    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"]
Oct 30 21:02:56.290956 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.290932    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.295267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.295240    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-8fa15b-openshift-default-dockercfg-fdxrv\""
Oct 30 21:02:56.304640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.304614    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"]
Oct 30 21:02:56.357694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.357661    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-envoy\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.357907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.357710    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-socket\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.357907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.357787    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-certs\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.357907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.357887    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-credential-socket\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.358077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.357935    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpwd9\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-kube-api-access-gpwd9\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.358077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.357979    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b830a023-a9ed-4708-90d2-e2a946db163c-istiod-ca-cert\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.358077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.358007    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-istio-token\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.358077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.358065    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-data\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.358291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.358138    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b830a023-a9ed-4708-90d2-e2a946db163c-istio-podinfo\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.416412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.413461    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:56.467409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467294    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-certs\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467405    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-credential-socket\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467436    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpwd9\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-kube-api-access-gpwd9\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467499    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b830a023-a9ed-4708-90d2-e2a946db163c-istiod-ca-cert\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467522    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-istio-token\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467569    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-data\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467663    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b830a023-a9ed-4708-90d2-e2a946db163c-istio-podinfo\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467785    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-envoy\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.467983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.467834    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-socket\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.468376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.468311    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-socket\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.485411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.468556    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-certs\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.485411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.468774    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-credential-socket\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.485411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.469703    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b830a023-a9ed-4708-90d2-e2a946db163c-istiod-ca-cert\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.489470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.489437    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-data\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.502075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.502038    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-istio-token\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.516460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.516370    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpwd9\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-kube-api-access-gpwd9\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.517031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.517004    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-envoy\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.517703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.517680    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b830a023-a9ed-4708-90d2-e2a946db163c-istio-podinfo\") pod \"e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") " pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.585794 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.585685    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:56.608813 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.608778    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:56.692410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.692369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:56.920458 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:02:56.920420    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb830a023_a9ed_4708_90d2_e2a946db163c.slice/crio-d1a8ee84e9b98e4204bc6658b28b56e81c84c385518655c7875ae4dd3da0d5d7 WatchSource:0}: Error finding container d1a8ee84e9b98e4204bc6658b28b56e81c84c385518655c7875ae4dd3da0d5d7: Status 404 returned error can't find the container with id d1a8ee84e9b98e4204bc6658b28b56e81c84c385518655c7875ae4dd3da0d5d7
Oct 30 21:02:56.920940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.920916    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"]
Oct 30 21:02:56.923248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.923213    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:56.923334 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.923284    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:56.923334 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:56.923316    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:02:57.114430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:57.114113    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:57.247490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:57.247211    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:57.398412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:57.388207    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:57.789344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:57.788099    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" event={"ID":"b830a023-a9ed-4708-90d2-e2a946db163c","Type":"ContainerStarted","Data":"a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23"}
Oct 30 21:02:57.789344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:57.788167    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" event={"ID":"b830a023-a9ed-4708-90d2-e2a946db163c","Type":"ContainerStarted","Data":"d1a8ee84e9b98e4204bc6658b28b56e81c84c385518655c7875ae4dd3da0d5d7"}
Oct 30 21:02:57.840477 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:57.826457    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" podStartSLOduration=1.826432141 podStartE2EDuration="1.826432141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:57.823937344 +0000 UTC m=+1485.938671533" watchObservedRunningTime="2026-10-30 21:02:57.826432141 +0000 UTC m=+1485.941166332"
Oct 30 21:02:58.210268 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:58.210183    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:58.312408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:58.310869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:58.410166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:58.410101    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:58.609735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:58.609696    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:02:58.611634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:58.611605    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:02:58.611786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:58.611657    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:02:59.135553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:59.135516    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:59.609588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:59.609550    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:02:59.609810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:59.609607    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:02:59.973931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:02:59.973819    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:00.586540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:00.585978    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:00.609884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:00.609848    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:03:00.610092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:00.609919    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:03:00.921648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:00.921289    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:00.940121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:00.940069    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"]
Oct 30 21:03:00.940692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:00.940647    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl" podUID="9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" containerName="maas-api" containerID="cri-o://cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf" gracePeriod=30
Oct 30 21:03:01.537205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.537179    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:03:01.609465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.609432    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:03:01.609901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.609494    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:03:01.619016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.618976    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:01.641915 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.641879    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-openshift-service-ca\") pod \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") "
Oct 30 21:03:01.642089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.642021    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-kube-api-access-wnb7l\" (UniqueName: \"kubernetes.io/projected/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-kube-api-access-wnb7l\") pod \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") "
Oct 30 21:03:01.642089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.642044    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-metrics-certs\") pod \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") "
Oct 30 21:03:01.642089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.642071    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-maas-api-tls\") pod \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\" (UID: \"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26\") "
Oct 30 21:03:01.642263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.642245    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-openshift-service-ca" pod "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" (UID: "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:03:01.642911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.642882    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:01.644364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.644336    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-kube-api-access-wnb7l" pod "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" (UID: "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26"). InnerVolumeSpecName "kube-api-access-wnb7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:03:01.644364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.644344    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-maas-api-tls" pod "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" (UID: "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:03:01.644534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.644450    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-metrics-certs" pod "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" (UID: "9dd45ccd-a11a-455d-bde1-5f03ac8a7a26"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:03:01.714971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.714926    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:01.744854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.744821    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnb7l\" (UniqueName: \"kubernetes.io/projected/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-kube-api-access-wnb7l\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:01.744854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.744852    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:01.744854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.744862    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:01.823359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.821763    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:01.923937 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.923768    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:01.949439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.949350    4852 generic.go:356] "Generic (PLEG): container finished" podID="9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" containerID="cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf" exitCode=0
Oct 30 21:03:01.949439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.949430    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"
Oct 30 21:03:01.949688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.949429    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl" event={"ID":"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26","Type":"ContainerDied","Data":"cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf"}
Oct 30 21:03:01.949688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.949633    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl" event={"ID":"9dd45ccd-a11a-455d-bde1-5f03ac8a7a26","Type":"ContainerDied","Data":"80bc2db8824f27d951fe378e01ad913fde2b9f951a573c8e2913e203476adc3b"}
Oct 30 21:03:01.949688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:01.949669    4852 scope.go:122] "RemoveContainer" containerID="cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf"
Oct 30 21:03:02.010472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.010449    4852 scope.go:122] "RemoveContainer" containerID="cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf"
Oct 30 21:03:02.011356 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:03:02.011324    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf\": container with ID starting with cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf not found: ID does not exist" containerID="cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf"
Oct 30 21:03:02.011462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.011365    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf"} err="failed to get container status \"cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf\": rpc error: code = NotFound desc = could not find container \"cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf\": container with ID starting with cdc770d38fabf8bdf4ea1cacc1ff03f176884851d375d3cba7aabbf7d6d5fbbf not found: ID does not exist"
Oct 30 21:03:02.019895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.019861    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"]
Oct 30 21:03:02.022315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.022283    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:02.026077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.026019    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-fa48f3-777684b557-lhrnl"]
Oct 30 21:03:02.313692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.313651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:02.609721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.609634    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:03:02.609721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:02.609691    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 21:03:03.310059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.310027    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"]
Oct 30 21:03:03.310324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.310300    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx" podUID="a73cd97e-3efc-4fb2-984e-b7d5386957c4" containerName="maas-api" containerID="cri-o://aa3bb846bbddf283cb123fbda4e5fe07bbd37d42513f2cc34ef652061e74e7c4" gracePeriod=30
Oct 30 21:03:03.344324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.344288    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:03.469690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.469657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:03.584162 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.584124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:03.668690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.665692    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:03:03.668690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.667264    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:03:03.772738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.728663    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:03:03.844529 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.832465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:03.951286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:03.951075    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:04.068003 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.067959    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:04.070105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.070036    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" path="/var/lib/kubelet/pods/9dd45ccd-a11a-455d-bde1-5f03ac8a7a26/volumes"
Oct 30 21:03:04.071824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.071790    4852 generic.go:356] "Generic (PLEG): container finished" podID="a73cd97e-3efc-4fb2-984e-b7d5386957c4" containerID="aa3bb846bbddf283cb123fbda4e5fe07bbd37d42513f2cc34ef652061e74e7c4" exitCode=0
Oct 30 21:03:04.073613 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.073583    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx" event={"ID":"a73cd97e-3efc-4fb2-984e-b7d5386957c4","Type":"ContainerDied","Data":"aa3bb846bbddf283cb123fbda4e5fe07bbd37d42513f2cc34ef652061e74e7c4"}
Oct 30 21:03:04.139995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.139965    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 21:03:04.179932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.179894    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-metrics-certs\") pod \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") "
Oct 30 21:03:04.180149 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.180014    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls\") pod \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") "
Oct 30 21:03:04.180149 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.180134    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a73cd97e-3efc-4fb2-984e-b7d5386957c4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73cd97e-3efc-4fb2-984e-b7d5386957c4-openshift-service-ca\") pod \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") "
Oct 30 21:03:04.180306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.180280    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a73cd97e-3efc-4fb2-984e-b7d5386957c4-kube-api-access-lr7sm\" (UniqueName: \"kubernetes.io/projected/a73cd97e-3efc-4fb2-984e-b7d5386957c4-kube-api-access-lr7sm\") pod \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\" (UID: \"a73cd97e-3efc-4fb2-984e-b7d5386957c4\") "
Oct 30 21:03:04.195171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.195134    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a73cd97e-3efc-4fb2-984e-b7d5386957c4-openshift-service-ca" pod "a73cd97e-3efc-4fb2-984e-b7d5386957c4" (UID: "a73cd97e-3efc-4fb2-984e-b7d5386957c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:03:04.198054 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.198016    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a73cd97e-3efc-4fb2-984e-b7d5386957c4-kube-api-access-lr7sm" pod "a73cd97e-3efc-4fb2-984e-b7d5386957c4" (UID: "a73cd97e-3efc-4fb2-984e-b7d5386957c4"). InnerVolumeSpecName "kube-api-access-lr7sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:03:04.198210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.198106    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-metrics-certs" pod "a73cd97e-3efc-4fb2-984e-b7d5386957c4" (UID: "a73cd97e-3efc-4fb2-984e-b7d5386957c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:03:04.200875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.200845    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls" pod "a73cd97e-3efc-4fb2-984e-b7d5386957c4" (UID: "a73cd97e-3efc-4fb2-984e-b7d5386957c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:03:04.275173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.275128    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:04.283438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.283161    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73cd97e-3efc-4fb2-984e-b7d5386957c4-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:04.283438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.283226    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr7sm\" (UniqueName: \"kubernetes.io/projected/a73cd97e-3efc-4fb2-984e-b7d5386957c4-kube-api-access-lr7sm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:04.283438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.283265    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:04.283438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.283303    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73cd97e-3efc-4fb2-984e-b7d5386957c4-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:04.564217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:04.564177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:05.034361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:05.034330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:05.082320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:05.082283    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"
Oct 30 21:03:05.082320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:05.082290    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx" event={"ID":"a73cd97e-3efc-4fb2-984e-b7d5386957c4","Type":"ContainerDied","Data":"582bb1a94ca43bf9557c8ce81515c8b0d539cf93f4042ba570438462f572a198"}
Oct 30 21:03:05.082692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:05.082373    4852 scope.go:122] "RemoveContainer" containerID="aa3bb846bbddf283cb123fbda4e5fe07bbd37d42513f2cc34ef652061e74e7c4"
Oct 30 21:03:05.144572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:05.144536    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"]
Oct 30 21:03:05.149459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:05.149426    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b46fb7-5c759846c9-8dvlx"]
Oct 30 21:03:05.805331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:05.805290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:06.013544 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:06.013508    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a73cd97e-3efc-4fb2-984e-b7d5386957c4" path="/var/lib/kubelet/pods/a73cd97e-3efc-4fb2-984e-b7d5386957c4/volumes"
Oct 30 21:03:06.622442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:06.621101    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:07.445959 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:07.445925    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:08.261505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:08.261469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:08.365803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:08.365771    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:08.466593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:08.466554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:09.549552 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:09.549520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:09.754642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:09.754592    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:10.224663 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:10.224623    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:10.348409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:10.341255    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:10.598179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:10.598135    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"]
Oct 30 21:03:10.598920 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:10.598887    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy" containerID="cri-o://9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443" gracePeriod=30
Oct 30 21:03:10.603954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:10.603922    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:11.149656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:11.149607    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:12.022910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:12.022879    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:12.799305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:12.799232    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:13.511245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:13.511212    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:14.245504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:14.245471    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:14.345362 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:14.345326    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:15.209565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:15.209527    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:15.692349 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:15.692304    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:16.119150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.116713    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"]
Oct 30 21:03:16.119150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.117465    4852 cpu_manager.go:414] "containerMap: removing container" podUID="a73cd97e-3efc-4fb2-984e-b7d5386957c4" containerName="maas-api"
Oct 30 21:03:16.119150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.117481    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a73cd97e-3efc-4fb2-984e-b7d5386957c4" containerName="maas-api"
Oct 30 21:03:16.119150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.117519    4852 cpu_manager.go:414] "containerMap: removing container" podUID="9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" containerName="maas-api"
Oct 30 21:03:16.119150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.117524    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" containerName="maas-api"
Oct 30 21:03:16.119150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.117657    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="a73cd97e-3efc-4fb2-984e-b7d5386957c4" containerName="maas-api"
Oct 30 21:03:16.119150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.117674    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dd45ccd-a11a-455d-bde1-5f03ac8a7a26" containerName="maas-api"
Oct 30 21:03:16.121624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.120509    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:03:16.121624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.121373    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.134911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.134018    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-3a65ec-openshift-default-dockercfg-4tr7g\""
Oct 30 21:03:16.142341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.142306    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"]
Oct 30 21:03:16.189977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.189889    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:16.223509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.223154    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-credential-socket\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.223509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.223260    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-data\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.223509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.223321    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istiod-ca-cert\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.223509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.223360    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-envoy\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.223509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.223414    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-token\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-token\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.225844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.224278    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-certs\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.225844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.224347    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-kube-api-access-6lc4z\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-kube-api-access-6lc4z\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.226086    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-socket\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.226258    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-podinfo\") pod \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\" (UID: \"934f053b-f20d-44b6-a98e-dbb8f2c767f8\") "
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.226718    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-credential-socket\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.226811    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plk97\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-kube-api-access-plk97\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.226923    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47aaf1c6-f339-46d3-b3d5-a65397293537-istiod-ca-cert\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.226994    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-data\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.227023    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-socket\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.227062    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-envoy\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.227181    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-certs\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.227222    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-token\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.227926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.227358    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-podinfo\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.229690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.229653    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-credential-socket" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:03:16.230036 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.230015    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-data" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:03:16.230526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.230506    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istiod-ca-cert" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:03:16.360823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.324916    4852 generic.go:356] "Generic (PLEG): container finished" podID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerID="9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443" exitCode=0
Oct 30 21:03:16.360823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.325061    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" event={"ID":"934f053b-f20d-44b6-a98e-dbb8f2c767f8","Type":"ContainerDied","Data":"9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443"}
Oct 30 21:03:16.360823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.325117    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f" event={"ID":"934f053b-f20d-44b6-a98e-dbb8f2c767f8","Type":"ContainerDied","Data":"4128ab4dcfa7c37eb2bfa959a1ba3b031342657a7609cc504348c390f10ef6af"}
Oct 30 21:03:16.360823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.325149    4852 scope.go:122] "RemoveContainer" containerID="9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443"
Oct 30 21:03:16.360823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.325376    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396000    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-podinfo\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396148    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-credential-socket\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396240    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plk97\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-kube-api-access-plk97\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396363    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47aaf1c6-f339-46d3-b3d5-a65397293537-istiod-ca-cert\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396444    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-data\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396469    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-socket\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396504    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-envoy\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396609    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-certs\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396645    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-token\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396744    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396760    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.396777    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.461064    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-credential-socket\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.462699    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47aaf1c6-f339-46d3-b3d5-a65397293537-istiod-ca-cert\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.463093    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-data\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.463455    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-socket\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.465666    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-socket" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.466435    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-certs\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.485777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.467487    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-certs" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:03:16.489290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.489203    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-podinfo\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.489921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.489803    4852 scope.go:122] "RemoveContainer" containerID="c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8"
Oct 30 21:03:16.541947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.534821    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-token" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:03:16.541947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.534846    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.541947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.534871    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.541947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.537745    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-token\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.541947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.537950    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-podinfo" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:03:16.541947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.538744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:16.541947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.539320    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-kube-api-access-6lc4z" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "kube-api-access-6lc4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:03:16.542811 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.542379    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plk97\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-kube-api-access-plk97\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.543556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.543481    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-envoy\") pod \"e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") " pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.544102 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.544070    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-envoy" pod "934f053b-f20d-44b6-a98e-dbb8f2c767f8" (UID: "934f053b-f20d-44b6-a98e-dbb8f2c767f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:03:16.578256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.577930    4852 scope.go:122] "RemoveContainer" containerID="9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443"
Oct 30 21:03:16.578400 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:03:16.578320    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443\": container with ID starting with 9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443 not found: ID does not exist" containerID="9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443"
Oct 30 21:03:16.578400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.578360    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443"} err="failed to get container status \"9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443\": rpc error: code = NotFound desc = could not find container \"9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443\": container with ID starting with 9f3a2676b6ba24a4abe658ac8c8aab563b94fce0c95d722ccb7ec1323df47443 not found: ID does not exist"
Oct 30 21:03:16.578509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.578413    4852 scope.go:122] "RemoveContainer" containerID="c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8"
Oct 30 21:03:16.578814 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:03:16.578780    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8\": container with ID starting with c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8 not found: ID does not exist" containerID="c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8"
Oct 30 21:03:16.578904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.578821    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8"} err="failed to get container status \"c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8\": rpc error: code = NotFound desc = could not find container \"c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8\": container with ID starting with c3b6653359be5ccf7cd9eef15fa8683c1728d17867b05a817a980f9f2f3749b8 not found: ID does not exist"
Oct 30 21:03:16.650407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.639120    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.650407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.639155    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.650407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.639172    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lc4z\" (UniqueName: \"kubernetes.io/projected/934f053b-f20d-44b6-a98e-dbb8f2c767f8-kube-api-access-6lc4z\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.650407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.639188    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934f053b-f20d-44b6-a98e-dbb8f2c767f8-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:03:16.685526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.685487    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"]
Oct 30 21:03:16.691481 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.689926    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-fa48f3-openshift-default-55fbd96d9c-7ll6f"]
Oct 30 21:03:16.744584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.744517    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:16.787838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:16.787800    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:17.064899 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:03:17.064868    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47aaf1c6_f339_46d3_b3d5_a65397293537.slice/crio-9529ba7ee9089b7488aba60969ce661b31270b56b9007e24a03ef86d912bbcba WatchSource:0}: Error finding container 9529ba7ee9089b7488aba60969ce661b31270b56b9007e24a03ef86d912bbcba: Status 404 returned error can't find the container with id 9529ba7ee9089b7488aba60969ce661b31270b56b9007e24a03ef86d912bbcba
Oct 30 21:03:17.065088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.065005    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"]
Oct 30 21:03:17.067875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.067836    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:17.068002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.067920    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:17.068002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.067963    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:17.149150 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:03:17.149118    4852 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47aaf1c6_f339_46d3_b3d5_a65397293537.slice/crio-d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:03:17.158970 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.158941    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:17.383566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.383473    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" event={"ID":"47aaf1c6-f339-46d3-b3d5-a65397293537","Type":"ContainerStarted","Data":"d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824"}
Oct 30 21:03:17.384154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.384126    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" event={"ID":"47aaf1c6-f339-46d3-b3d5-a65397293537","Type":"ContainerStarted","Data":"9529ba7ee9089b7488aba60969ce661b31270b56b9007e24a03ef86d912bbcba"}
Oct 30 21:03:17.442040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.441955    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podStartSLOduration=1.441936857 podStartE2EDuration="1.441936857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:03:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:03:17.426636782 +0000 UTC m=+1505.541370974" watchObservedRunningTime="2026-10-30 21:03:17.441936857 +0000 UTC m=+1505.556671047"
Oct 30 21:03:17.612451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.612350    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:17.746415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:17.745316    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:18.009207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:18.009175    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" path="/var/lib/kubelet/pods/934f053b-f20d-44b6-a98e-dbb8f2c767f8/volumes"
Oct 30 21:03:18.477914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:18.477800    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:18.752340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:18.752252    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:19.402779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:19.402732    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:19.404001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:19.403973    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:03:19.428584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:19.428539    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:19.738029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:19.737944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:19.874625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:19.874590    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:20.405399 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:20.405359    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:21.083875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.083837    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:21.172547 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.172472    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"]
Oct 30 21:03:21.173091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.173078    4852 cpu_manager.go:414] "containerMap: removing container" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy"
Oct 30 21:03:21.173135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.173093    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy"
Oct 30 21:03:21.173135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.173123    4852 cpu_manager.go:414] "containerMap: removing container" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy"
Oct 30 21:03:21.173135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.173130    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy"
Oct 30 21:03:21.173286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.173275    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy"
Oct 30 21:03:21.173327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.173288    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="934f053b-f20d-44b6-a98e-dbb8f2c767f8" containerName="istio-proxy"
Oct 30 21:03:21.175892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.175872    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.176826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.176803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:21.185882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.185855    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-3a65ec\""
Oct 30 21:03:21.186137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.186112    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-3a65ec-dockercfg-wz56s\""
Oct 30 21:03:21.191693 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.191668    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"]
Oct 30 21:03:21.213556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.213524    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"]
Oct 30 21:03:21.216824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.216806    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.226542 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.226516    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"]
Oct 30 21:03:21.240587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.240554    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6d83db6-f575-405f-b80b-4c65738bfb48-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.240762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.240696    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws7j6\" (UniqueName: \"kubernetes.io/projected/c6d83db6-f575-405f-b80b-4c65738bfb48-kube-api-access-ws7j6\") pod \"payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.418410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.409572    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6d83db6-f575-405f-b80b-4c65738bfb48-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.418410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.409662    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glb6k\" (UniqueName: \"kubernetes.io/projected/bd7151ac-50e1-411b-a52f-f65a6e246b94-kube-api-access-glb6k\") pod \"payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.418410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.409756    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws7j6\" (UniqueName: \"kubernetes.io/projected/c6d83db6-f575-405f-b80b-4c65738bfb48-kube-api-access-ws7j6\") pod \"payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.418410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.409882    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd7151ac-50e1-411b-a52f-f65a6e246b94-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.418410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.416860    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6d83db6-f575-405f-b80b-4c65738bfb48-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.459272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.458254    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws7j6\" (UniqueName: \"kubernetes.io/projected/c6d83db6-f575-405f-b80b-4c65738bfb48-kube-api-access-ws7j6\") pod \"payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.469007 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.468964    4852 generic.go:356] "Generic (PLEG): container finished" podID="3c4f7322-e13b-41bc-b47b-39390953298e" containerID="55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52" exitCode=0
Oct 30 21:03:21.472357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.469183    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" event={"ID":"3c4f7322-e13b-41bc-b47b-39390953298e","Type":"ContainerDied","Data":"55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52"}
Oct 30 21:03:21.472357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.470318    4852 scope.go:122] "RemoveContainer" containerID="55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52"
Oct 30 21:03:21.474770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.473768    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:21.474770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.473925    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:21.474770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.474051    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:21.487883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.482061    4852 generic.go:356] "Generic (PLEG): container finished" podID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerID="58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182" exitCode=0
Oct 30 21:03:21.487883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.482621    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" event={"ID":"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1","Type":"ContainerDied","Data":"58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182"}
Oct 30 21:03:21.487883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.484075    4852 scope.go:122] "RemoveContainer" containerID="58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182"
Oct 30 21:03:21.489891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.489866    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:21.491313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.491275    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:21.491495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.491471    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:21.491564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.491528    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:03:21.527262 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.524420    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glb6k\" (UniqueName: \"kubernetes.io/projected/bd7151ac-50e1-411b-a52f-f65a6e246b94-kube-api-access-glb6k\") pod \"payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.527262 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.524760    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd7151ac-50e1-411b-a52f-f65a6e246b94-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.528769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.528741    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd7151ac-50e1-411b-a52f-f65a6e246b94-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.561210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.561147    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glb6k\" (UniqueName: \"kubernetes.io/projected/bd7151ac-50e1-411b-a52f-f65a6e246b94-kube-api-access-glb6k\") pod \"payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.575342 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:03:21.575309    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:03:21.665965 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.665820    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:21.839941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.839909    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:21.877835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:21.877809    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"]
Oct 30 21:03:22.189702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.189657    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"]
Oct 30 21:03:22.517412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.511221    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" event={"ID":"3c4f7322-e13b-41bc-b47b-39390953298e","Type":"ContainerStarted","Data":"a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e"}
Oct 30 21:03:22.536962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.536923    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" event={"ID":"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1","Type":"ContainerStarted","Data":"35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c"}
Oct 30 21:03:22.583324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.581237    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" event={"ID":"c6d83db6-f575-405f-b80b-4c65738bfb48","Type":"ContainerStarted","Data":"335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122"}
Oct 30 21:03:22.583324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.581315    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" event={"ID":"c6d83db6-f575-405f-b80b-4c65738bfb48","Type":"ContainerStarted","Data":"6d0c59facccf5bd261dfbcf2e50578b533fa2a66abebfaba073c3c172a96bbae"}
Oct 30 21:03:22.583324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.581496    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:22.611412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.597898    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:22.636406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.636348    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" event={"ID":"bd7151ac-50e1-411b-a52f-f65a6e246b94","Type":"ContainerStarted","Data":"18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca"}
Oct 30 21:03:22.636714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.636682    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" event={"ID":"bd7151ac-50e1-411b-a52f-f65a6e246b94","Type":"ContainerStarted","Data":"0b2e526e93e86a80d74ca08c6cc3675ba765e2b62ab2476577b2e07f71145917"}
Oct 30 21:03:22.647548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.647513    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:22.666005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.665917    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" podStartSLOduration=1.665895156 podStartE2EDuration="1.665895156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:03:22.646919614 +0000 UTC m=+1510.761653801" watchObservedRunningTime="2026-10-30 21:03:22.665895156 +0000 UTC m=+1510.780629346"
Oct 30 21:03:22.734450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:22.734364    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" podStartSLOduration=1.734346242 podStartE2EDuration="1.734346242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:03:22.668170194 +0000 UTC m=+1510.782904385" watchObservedRunningTime="2026-10-30 21:03:22.734346242 +0000 UTC m=+1510.849080433"
Oct 30 21:03:23.284413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:23.284269    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:03:23.284983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:23.284465    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:03:23.284983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:23.284515    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:03:23.491408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:23.490752    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:03:23.499938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:23.494495    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:03:23.499938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:23.494550    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:03:23.728207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:23.728114    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:24.290417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:24.289579    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:03:24.290417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:24.289645    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:03:24.497413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:24.493833    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:03:24.497413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:24.493906    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:03:24.588472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:24.585128    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:24.721411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:24.719526    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:03:25.297423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:25.296489    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:03:25.297423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:25.296555    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:03:25.327247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:25.327174    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:25.489412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:25.488749    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:03:25.489412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:25.488815    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:03:26.176407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:26.175911    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:26.270412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:26.267526    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:26.288756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:26.285768    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:03:26.288756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:26.285833    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:03:26.489471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:26.488825    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:03:26.489471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:26.488890    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:03:26.740097 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:26.739985    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:03:27.129453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:27.129401    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:27.284964 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:27.284929    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 21:03:27.285249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:27.285221    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 21:03:27.501464 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:27.500660    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused" start-of-body=
Oct 30 21:03:27.501464 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:27.500721    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.130:15021/healthz/ready\": dial tcp 10.133.0.130:15021: connect: connection refused"
Oct 30 21:03:28.127609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:28.127578    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:28.407824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:28.407744    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:03:28.410543 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:28.410513    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:03:28.494193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:28.494156    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:03:28.496307 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:28.496281    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:03:29.123138 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:29.123098    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:29.809665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:29.809627    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:30.520512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:30.520477    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:31.212038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:31.211999    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:31.313033 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:31.312992    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:31.390530 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:31.390487    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:31.477738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:31.477653    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:31.699272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:31.699245    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:32.387111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:32.387077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:33.452571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:33.452536    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:33.558248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:33.558208    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:34.732011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:34.731984    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:03:34.734347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:34.734323    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:03:34.753643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:34.753610    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:35.604729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:35.604692    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:36.567976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:36.567931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:37.641805 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:37.641767    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:38.768517 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:38.768478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:39.842365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:39.842331    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:40.584007 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:40.583971    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:41.443993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:41.443959    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:42.368502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:42.368472    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:42.806495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:42.806460    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:43.473870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:43.473833    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:44.450734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:44.450695    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:45.241641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:45.241593    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:46.601555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:46.601515    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:46.889924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:46.889832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:48.280299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:48.280259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:48.668548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:48.668447    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:49.770614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:49.770570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:50.161468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:50.158014    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:50.575974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:50.575936    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:50.829746 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:50.829663    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:51.092118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:51.092022    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:51.192229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:51.192188    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:51.581895 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:03:51.581846    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:03:52.218986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:52.218955    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:53.238791 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:53.238751    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:54.190035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:54.189993    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:54.882637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:54.882600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:55.164766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:55.164680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:55.369891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:55.369840    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:56.079458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.079414    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:56.177507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.177470    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"]
Oct 30 21:03:56.210581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.209492    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:56.219615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.218717    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-a50675-dockercfg-qf985\""
Oct 30 21:03:56.219615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.219288    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-a50675\""
Oct 30 21:03:56.219615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.219540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:56.228468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.228434    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"]
Oct 30 21:03:56.234428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.234326    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"]
Oct 30 21:03:56.242494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.242468    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"]
Oct 30 21:03:56.242623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.242564    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.508002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.507970    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:56.512762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.512733    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fglmr\" (UniqueName: \"kubernetes.io/projected/7937d42d-4d76-4908-a189-b3e03aa65932-kube-api-access-fglmr\") pod \"payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.513042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.513020    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0162ea80-1b44-4504-a24d-26eefcbdfe5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:56.513199 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.513167    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7937d42d-4d76-4908-a189-b3e03aa65932-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.513351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.513335    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6zkq\" (UniqueName: \"kubernetes.io/projected/0162ea80-1b44-4504-a24d-26eefcbdfe5b-kube-api-access-j6zkq\") pod \"payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:56.605256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.605159    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:56.614710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.614671    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fglmr\" (UniqueName: \"kubernetes.io/projected/7937d42d-4d76-4908-a189-b3e03aa65932-kube-api-access-fglmr\") pod \"payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.614929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.614753    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0162ea80-1b44-4504-a24d-26eefcbdfe5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:56.614929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.614830    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7937d42d-4d76-4908-a189-b3e03aa65932-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.614929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.614899    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6zkq\" (UniqueName: \"kubernetes.io/projected/0162ea80-1b44-4504-a24d-26eefcbdfe5b-kube-api-access-j6zkq\") pod \"payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:56.615838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.615814    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0162ea80-1b44-4504-a24d-26eefcbdfe5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:56.616200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.616170    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7937d42d-4d76-4908-a189-b3e03aa65932-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.625144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.625110    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6zkq\" (UniqueName: \"kubernetes.io/projected/0162ea80-1b44-4504-a24d-26eefcbdfe5b-kube-api-access-j6zkq\") pod \"payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:56.625323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.625253    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fglmr\" (UniqueName: \"kubernetes.io/projected/7937d42d-4d76-4908-a189-b3e03aa65932-kube-api-access-fglmr\") pod \"payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.689247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.689214    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:56.721324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.721278    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"]
Oct 30 21:03:56.722660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.722621    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy" containerID="cri-o://b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137" gracePeriod=30
Oct 30 21:03:56.744455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.743964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:56.890775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:56.890735    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:57.224876 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:03:57.224843    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7937d42d_4d76_4908_a189_b3e03aa65932.slice/crio-37d145b0096b3251620063f213e9b1a7d04bc90a034288bfab7f8b2e5e81d954 WatchSource:0}: Error finding container 37d145b0096b3251620063f213e9b1a7d04bc90a034288bfab7f8b2e5e81d954: Status 404 returned error can't find the container with id 37d145b0096b3251620063f213e9b1a7d04bc90a034288bfab7f8b2e5e81d954
Oct 30 21:03:57.225254 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:57.224947    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"]
Oct 30 21:03:57.271153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:57.271121    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"]
Oct 30 21:03:57.273030 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:03:57.272996    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0162ea80_1b44_4504_a24d_26eefcbdfe5b.slice/crio-51ed5d8449794bbc3fb97667e9741305dcdd6e51070ea199feba3986861977c0 WatchSource:0}: Error finding container 51ed5d8449794bbc3fb97667e9741305dcdd6e51070ea199feba3986861977c0: Status 404 returned error can't find the container with id 51ed5d8449794bbc3fb97667e9741305dcdd6e51070ea199feba3986861977c0
Oct 30 21:03:57.309685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:57.309654    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:58.133590 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.133549    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" event={"ID":"0162ea80-1b44-4504-a24d-26eefcbdfe5b","Type":"ContainerStarted","Data":"3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09"}
Oct 30 21:03:58.133921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.133622    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" event={"ID":"0162ea80-1b44-4504-a24d-26eefcbdfe5b","Type":"ContainerStarted","Data":"51ed5d8449794bbc3fb97667e9741305dcdd6e51070ea199feba3986861977c0"}
Oct 30 21:03:58.133921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.133655    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:03:58.135331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.135304    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" event={"ID":"7937d42d-4d76-4908-a189-b3e03aa65932","Type":"ContainerStarted","Data":"bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d"}
Oct 30 21:03:58.135486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.135347    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" event={"ID":"7937d42d-4d76-4908-a189-b3e03aa65932","Type":"ContainerStarted","Data":"37d145b0096b3251620063f213e9b1a7d04bc90a034288bfab7f8b2e5e81d954"}
Oct 30 21:03:58.135486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.135369    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:03:58.154502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.154453    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" podStartSLOduration=2.15444064 podStartE2EDuration="2.15444064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:03:58.151508492 +0000 UTC m=+1546.266242682" watchObservedRunningTime="2026-10-30 21:03:58.15444064 +0000 UTC m=+1546.269174829"
Oct 30 21:03:58.174576 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.174518    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" podStartSLOduration=2.174498043 podStartE2EDuration="2.174498043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:03:58.170999972 +0000 UTC m=+1546.285734170" watchObservedRunningTime="2026-10-30 21:03:58.174498043 +0000 UTC m=+1546.289232235"
Oct 30 21:03:58.259034 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.258966    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:58.461128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:58.461040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:59.146671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:03:59.146636    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:00.061625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.061593    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:00.153213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.153174    4852 generic.go:356] "Generic (PLEG): container finished" podID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerID="d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824" exitCode=0
Oct 30 21:04:00.153467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.153310    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" event={"ID":"47aaf1c6-f339-46d3-b3d5-a65397293537","Type":"ContainerDied","Data":"d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824"}
Oct 30 21:04:00.153893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.153874    4852 scope.go:122] "RemoveContainer" containerID="d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824"
Oct 30 21:04:00.155597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.155570    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:00.155678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.155639    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:00.155678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.155670    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:00.943709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:00.943677    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:01.178796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:01.178756    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" event={"ID":"47aaf1c6-f339-46d3-b3d5-a65397293537","Type":"ContainerStarted","Data":"8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2"}
Oct 30 21:04:01.745192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:01.745161    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:04:01.746436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:01.746403    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:04:01.746562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:01.746469    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 21:04:01.801697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:01.801662    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:02.116088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.116049    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:04:02.191063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.191025    4852 generic.go:356] "Generic (PLEG): container finished" podID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerID="b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137" exitCode=0
Oct 30 21:04:02.191653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.191093    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"
Oct 30 21:04:02.191653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.191094    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" event={"ID":"7129cda6-8378-4d30-b0e4-7a7bcbcd6196","Type":"ContainerDied","Data":"b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137"}
Oct 30 21:04:02.191653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.191216    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259" event={"ID":"7129cda6-8378-4d30-b0e4-7a7bcbcd6196","Type":"ContainerDied","Data":"0ae840c635d3d5d7285369b5ee564bb5b56d86969d2e05e972504595b6bf53bc"}
Oct 30 21:04:02.191653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.191242    4852 scope.go:122] "RemoveContainer" containerID="b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137"
Oct 30 21:04:02.214595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.214568    4852 scope.go:122] "RemoveContainer" containerID="b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137"
Oct 30 21:04:02.217550 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:04:02.217525    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137\": container with ID starting with b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137 not found: ID does not exist" containerID="b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137"
Oct 30 21:04:02.217654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.217561    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137"} err="failed to get container status \"b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137\": rpc error: code = NotFound desc = could not find container \"b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137\": container with ID starting with b0b8e83c1ac37cefd456f3ab4d19a7efa3b47692393055dcd87cec74760ea137 not found: ID does not exist"
Oct 30 21:04:02.241488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.241456    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-socket\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.241656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.241504    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-token\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-token\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.241656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.241547    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-data\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.241773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.241672    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-envoy\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.241773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.241742    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-socket" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:04:02.241882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.241830    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-data" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:04:02.242043 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242024    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-credential-socket\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.242157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242143    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-certs\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.242255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242210    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istiod-ca-cert\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.242312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242288    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-credential-socket" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:04:02.242366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242312    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-podinfo\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.242440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242418    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-certs" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:04:02.242497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242445    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-kube-api-access-4vwx5\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-kube-api-access-4vwx5\") pod \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\" (UID: \"7129cda6-8378-4d30-b0e4-7a7bcbcd6196\") "
Oct 30 21:04:02.242712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.242683    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istiod-ca-cert" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:04:02.243250 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.243223    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.243347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.243254    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.243347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.243270    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.243347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.243285    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.243347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.243299    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.244128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.244098    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-envoy" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:04:02.244859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.244824    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-podinfo" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:04:02.245252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.245230    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-kube-api-access-4vwx5" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "kube-api-access-4vwx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:04:02.245342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.245285    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-token" pod "7129cda6-8378-4d30-b0e4-7a7bcbcd6196" (UID: "7129cda6-8378-4d30-b0e4-7a7bcbcd6196"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:04:02.344514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.344411    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.344514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.344447    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vwx5\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-kube-api-access-4vwx5\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.344514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.344463    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.344514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.344475    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7129cda6-8378-4d30-b0e4-7a7bcbcd6196-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:04:02.519010 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.518974    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"]
Oct 30 21:04:02.525650 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.525606    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-b46fb7-openshift-default-99ccd7957-mk259"]
Oct 30 21:04:02.623056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.622966    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:02.747508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.746929    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:04:02.747508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:02.747008    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 21:04:03.392782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:03.392749    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:03.745019 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:03.744930    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:04:03.745019 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:03.744994    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 21:04:04.014799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:04.014761    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" path="/var/lib/kubelet/pods/7129cda6-8378-4d30-b0e4-7a7bcbcd6196/volumes"
Oct 30 21:04:04.092378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:04.092343    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:04.199696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:04.199657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:04.403322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:04.403232    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:04:04.745843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:04.745765    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:04:04.745843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:04.745820    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 21:04:04.867420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:04.867363    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:05.756892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:05.756852    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:04:05.757533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:05.757500    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 21:04:06.082318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:06.082208    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:06.745458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:06.745421    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:04:06.745654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:06.745488    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.134:15021/healthz/ready\": dial tcp 10.133.0.134:15021: connect: connection refused"
Oct 30 21:04:06.843564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:06.843528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:06.947457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:06.947418    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:07.049015 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:07.048983    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:07.747759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:07.747719    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:07.750827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:07.750804    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:04:07.752048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:07.752029    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:04:07.846666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:07.846626    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:08.618288 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:08.618233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:09.147276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:09.147246    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:04:09.147729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:09.147328    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:04:09.351617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:09.351582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:09.976692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:09.976649    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:10.088238 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:10.088208    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:10.188338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:10.188308    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:10.290256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:10.290218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:10.465502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:10.465469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:10.636951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:10.636859    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:11.018302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:11.018258    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:11.634718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:11.634661    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:12.337648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:12.337610    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:12.834569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:12.834537    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:12.934556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:12.934525    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:13.130589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:13.130513    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:13.922309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:13.922267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:14.783907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:14.783863    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:15.868571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:15.868538    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:16.750477 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:16.750445    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:17.520884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:17.520850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:17.633792 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:17.633752    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:17.736917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:17.736876    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:17.863427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:17.863321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:18.629712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:18.629680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:19.491651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:19.491613    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:19.597628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:19.597586    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:19.960780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:19.960738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:20.122415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:20.109061    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:20.982285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:20.982253    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:21.559771 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:04:21.559735    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:04:21.931332 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:21.931250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:22.877158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:22.877119    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:23.451361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:23.451321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:23.552461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:23.552425    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:23.926063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:23.926033    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:24.676132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:24.676084    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:24.894007 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:24.893965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:25.209518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:25.209484    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:25.552005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:25.551970    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:26.461923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:26.461871    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:27.108598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:27.108562    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:27.493493 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:27.493401    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:28.018582 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:28.018549    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:28.821302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:28.821272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:29.752636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:29.752603    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:30.537485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:30.537443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:31.351138 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:31.347934    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:32.136056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:32.136021    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:32.934257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:32.934218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:33.988449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:33.988412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.289188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:35.289150    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.390807 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:35.390696    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:35.673159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:35.673076    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:36.663479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:36.663369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:37.647274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:37.647239    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:38.742315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:38.742271    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:39.718735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:39.718699    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:40.750789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:40.750750    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:41.526289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:41.526258    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:41.626337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:41.626304    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:41.728153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:41.728116    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:42.592308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:42.592259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:43.451935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:43.451895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:43.872321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:43.872285    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:44.537557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:44.537521    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:45.389424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:45.389394    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:46.173401 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:46.173358    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:46.864148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:46.864107    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:47.740449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:47.740411    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:48.621858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:48.621826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:49.502677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:49.502638    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:50.588806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:50.588767    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:51.457945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:51.457900    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:51.562740 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:04:51.562702    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:04:52.285986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:52.285953    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:53.132640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:53.132602    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:53.860529 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:53.860470    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:53.960184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:53.960140    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:54.743561 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:54.743519    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:55.084847 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.084812    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:55.510692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.510652    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"]
Oct 30 21:04:55.511301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.511288    4852 cpu_manager.go:414] "containerMap: removing container" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy"
Oct 30 21:04:55.511361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.511303    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy"
Oct 30 21:04:55.511452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.511442    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="7129cda6-8378-4d30-b0e4-7a7bcbcd6196" containerName="istio-proxy"
Oct 30 21:04:55.514437 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.514419    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.523132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.523105    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-49c7c2-openshift-default-dockercfg-n8mj6\""
Oct 30 21:04:55.527714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.527691    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"]
Oct 30 21:04:55.540814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.540785    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-data\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.540965 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.540830    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-socket\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.540965 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.540941    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-envoy\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.541052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.540974    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbggq\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-kube-api-access-dbggq\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.541052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.541033    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-token\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.541120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.541103    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-podinfo\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.541299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.541282    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-certs\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.541343 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.541334    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istiod-ca-cert\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.541423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.541373    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-credential-socket\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.544828 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.544803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:55.643119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643086    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-token\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643159    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-podinfo\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643332    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-certs\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643372    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istiod-ca-cert\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643442    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-credential-socket\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643501    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-data\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643543    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-socket\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643654    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-envoy\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643693    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbggq\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-kube-api-access-dbggq\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643745    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-certs\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643784    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-credential-socket\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643875    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-data\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.643997 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.643977    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-socket\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.644343 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.644325    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istiod-ca-cert\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.645032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.645009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:55.645972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.645952    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-envoy\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.646056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.645952    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-podinfo\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.651715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.651695    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-token\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.651833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.651821    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbggq\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-kube-api-access-dbggq\") pod \"e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") " pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:55.832696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:55.832618    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:56.182897 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:04:56.182867    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fdcb224_4ee2_41da_942a_6298b75d1a0b.slice/crio-c759b35237d5692ad0f6dd10df388a3afb5f0d2ad48f493e2a75d19fa982cd26 WatchSource:0}: Error finding container c759b35237d5692ad0f6dd10df388a3afb5f0d2ad48f493e2a75d19fa982cd26: Status 404 returned error can't find the container with id c759b35237d5692ad0f6dd10df388a3afb5f0d2ad48f493e2a75d19fa982cd26
Oct 30 21:04:56.182897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.182884    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"]
Oct 30 21:04:56.185432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.185404    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:56.185520 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.185465    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:56.185520 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.185491    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:56.453582 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.453503    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:56.822016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.821979    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" event={"ID":"8fdcb224-4ee2-41da-942a-6298b75d1a0b","Type":"ContainerStarted","Data":"096e63de71a8bfc2c1acd651e381fae1784565d91b64056c23e721155d93d204"}
Oct 30 21:04:56.822232 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.822042    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" event={"ID":"8fdcb224-4ee2-41da-942a-6298b75d1a0b","Type":"ContainerStarted","Data":"c759b35237d5692ad0f6dd10df388a3afb5f0d2ad48f493e2a75d19fa982cd26"}
Oct 30 21:04:56.833591 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.833556    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:56.844343 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:56.844292    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podStartSLOduration=1.844278606 podStartE2EDuration="1.844278606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:04:56.843354339 +0000 UTC m=+1604.958088529" watchObservedRunningTime="2026-10-30 21:04:56.844278606 +0000 UTC m=+1604.959012796"
Oct 30 21:04:57.262380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.262346    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"]
Oct 30 21:04:57.266986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.266962    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.269510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.269486    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:57.275333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.275287    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-321ccd-openshift-default-dockercfg-f2b7f\""
Oct 30 21:04:57.278310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.278284    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"]
Oct 30 21:04:57.415609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.415571    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-socket\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.415832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.415781    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4gjc\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-kube-api-access-j4gjc\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.415832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.415817    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-certs\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.415963 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.415846    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-podinfo\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.416110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.416085    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a8ec2e-6164-49dd-98d4-a23312d456f2-istiod-ca-cert\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.416183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.416164    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-data\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.416241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.416192    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-token\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.416287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.416271    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-credential-socket\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.416353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.416336    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-envoy\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518199    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4gjc\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-kube-api-access-j4gjc\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518245    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-certs\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518267    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-podinfo\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518372    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a8ec2e-6164-49dd-98d4-a23312d456f2-istiod-ca-cert\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518446    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-data\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518469    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-token\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518510    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-credential-socket\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518584    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-envoy\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518648    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-socket\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518844    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-data\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.518960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518871    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-certs\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.519005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.518957    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-credential-socket\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.519129 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.519105    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-socket\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.519321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.519302    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a8ec2e-6164-49dd-98d4-a23312d456f2-istiod-ca-cert\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.520887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.520867    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-envoy\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.521118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.521097    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-podinfo\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.526414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.526335    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-token\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.526551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.526454    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4gjc\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-kube-api-access-j4gjc\") pod \"e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") " pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.584791 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.584749    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:57.840091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.840005    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:57.873584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.873554    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"]
Oct 30 21:04:57.874430 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:04:57.874399    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a8ec2e_6164_49dd_98d4_a23312d456f2.slice/crio-0a9b0fb28db3fb6a5903b38b28fc1a1077a09ab3b43a3a7300d25f90497c42dc WatchSource:0}: Error finding container 0a9b0fb28db3fb6a5903b38b28fc1a1077a09ab3b43a3a7300d25f90497c42dc: Status 404 returned error can't find the container with id 0a9b0fb28db3fb6a5903b38b28fc1a1077a09ab3b43a3a7300d25f90497c42dc
Oct 30 21:04:57.877633 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.877597    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:57.877738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.877718    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:57.877781 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:57.877768    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:04:58.066702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:58.066670    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:58.878086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:58.878049    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" event={"ID":"71a8ec2e-6164-49dd-98d4-a23312d456f2","Type":"ContainerStarted","Data":"bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d"}
Oct 30 21:04:58.882908 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:58.882886    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:58.883078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:58.883058    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" event={"ID":"71a8ec2e-6164-49dd-98d4-a23312d456f2","Type":"ContainerStarted","Data":"0a9b0fb28db3fb6a5903b38b28fc1a1077a09ab3b43a3a7300d25f90497c42dc"}
Oct 30 21:04:58.884181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:58.884158    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:04:58.913179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:58.913116    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podStartSLOduration=1.913096215 podStartE2EDuration="1.913096215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:04:58.903846112 +0000 UTC m=+1607.018580303" watchObservedRunningTime="2026-10-30 21:04:58.913096215 +0000 UTC m=+1607.027830405"
Oct 30 21:04:59.128511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:59.128353    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:59.251235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:59.251166    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:59.585804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:59.585769    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:04:59.587595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:59.587568    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:04:59.587671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:04:59.587622    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:05:00.511017 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.510977    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:00.586089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.586052    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:05:00.586297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.586136    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:05:00.628782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.628742    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"]
Oct 30 21:05:00.636622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.636582    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:00.642139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.642109    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:00.642426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.642190    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"]
Oct 30 21:05:00.707801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.707767    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-49c7c2\""
Oct 30 21:05:00.708163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.708141    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-49c7c2-dockercfg-cmgdm\""
Oct 30 21:05:00.775466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.755558    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4097963-cd6c-416a-b98d-83bcb875d7b2-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:00.775466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.755626    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4z4p\" (UniqueName: \"kubernetes.io/projected/f4097963-cd6c-416a-b98d-83bcb875d7b2-kube-api-access-r4z4p\") pod \"payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:00.885411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.862489    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4097963-cd6c-416a-b98d-83bcb875d7b2-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:00.885411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.862553    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4z4p\" (UniqueName: \"kubernetes.io/projected/f4097963-cd6c-416a-b98d-83bcb875d7b2-kube-api-access-r4z4p\") pod \"payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:00.885411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:00.863895    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4097963-cd6c-416a-b98d-83bcb875d7b2-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:01.017725 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:01.008088    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4z4p\" (UniqueName: \"kubernetes.io/projected/f4097963-cd6c-416a-b98d-83bcb875d7b2-kube-api-access-r4z4p\") pod \"payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:01.017725 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:01.017432    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:01.034827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:01.034767    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:01.412287 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:05:01.412237    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4097963_cd6c_416a_b98d_83bcb875d7b2.slice/crio-1e4d80c04b51f8caced66750de857e5e82fc748f202a348fdc26db09ea06dec1 WatchSource:0}: Error finding container 1e4d80c04b51f8caced66750de857e5e82fc748f202a348fdc26db09ea06dec1: Status 404 returned error can't find the container with id 1e4d80c04b51f8caced66750de857e5e82fc748f202a348fdc26db09ea06dec1
Oct 30 21:05:01.412881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:01.412816    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"]
Oct 30 21:05:01.585975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:01.585934    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:05:01.586375 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:01.586001    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:05:02.028663 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.028622    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l" event={"ID":"f4097963-cd6c-416a-b98d-83bcb875d7b2","Type":"ContainerStarted","Data":"a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd"}
Oct 30 21:05:02.028864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.028689    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l" event={"ID":"f4097963-cd6c-416a-b98d-83bcb875d7b2","Type":"ContainerStarted","Data":"1e4d80c04b51f8caced66750de857e5e82fc748f202a348fdc26db09ea06dec1"}
Oct 30 21:05:02.028864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.028760    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:02.073412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.073282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:02.080319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.080259    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l" podStartSLOduration=2.080243326 podStartE2EDuration="2.080243326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:05:02.071377946 +0000 UTC m=+1610.186112137" watchObservedRunningTime="2026-10-30 21:05:02.080243326 +0000 UTC m=+1610.194977531"
Oct 30 21:05:02.585119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.585088    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:05:02.585301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.585143    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:05:02.939977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:02.939946    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:03.585869 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:03.585830    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:05:03.586058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:03.585904    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:05:03.945319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:03.945231    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:04.175976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:04.175937    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:04.822586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:04.822546    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:05:04.823134 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:04.823108    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:05:04.824620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:04.824451    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:05:04.923776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:04.923740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:05.729189 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:05.729154    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:06.619046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:06.619010    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:07.383380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:07.383340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.120435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:08.120401    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:08.957715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:08.957687    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:09.728971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:09.728938    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:10.518617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:10.518586    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:11.242255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:11.242222    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.205140 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:12.205103    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.318111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:12.318073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.412242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:12.412206    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:12.822029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:12.821988    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:13.040690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:13.040660    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:05:13.254013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:13.253895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:14.169605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:14.169572    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:15.055208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:15.055172    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:15.878673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:15.878632    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:16.735694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:16.735649    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:17.646149 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:17.646113    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:18.356426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:18.356322    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:19.128012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:19.127965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:20.018227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:20.018191    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:20.732222 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:20.732181    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:21.497463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:21.497422    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:21.574478 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:05:21.574426    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:05:22.176310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:22.176251    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:22.877699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:22.877663    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:23.605253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:23.605210    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:24.332471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:24.332441    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:25.031738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:25.031696    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:26.083546 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:26.083514    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:26.938490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:26.938456    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:27.537121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:27.537076    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"]
Oct 30 21:05:27.537637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:27.537340    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy" containerID="cri-o://21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d" gracePeriod=30
Oct 30 21:05:27.641843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:27.641811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:28.517608 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:28.517567    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:29.427435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:29.427363    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:29.515480 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:29.514776    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:30.212851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:30.212812    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:30.931084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:30.931053    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:31.851161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:31.851105    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:32.030358 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.030315    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:32.711464 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.711415    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:32.856137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.853411    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 21:05:32.903093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903044    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-certs\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903131    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-socket\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903247    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75a626ea-7131-4ae2-a793-c566838aec17-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a626ea-7131-4ae2-a793-c566838aec17-istio-podinfo\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903345    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-certs" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:05:32.903486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903370    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-credential-socket\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903446    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-kube-api-access-l5z98\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-kube-api-access-l5z98\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903474    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-socket" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:05:32.903718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903492    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-istio-token\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-istio-token\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903569    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-data\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903633    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-credential-socket" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:05:32.903718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903659    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-envoy\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903709    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75a626ea-7131-4ae2-a793-c566838aec17-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a626ea-7131-4ae2-a793-c566838aec17-istiod-ca-cert\") pod \"75a626ea-7131-4ae2-a793-c566838aec17\" (UID: \"75a626ea-7131-4ae2-a793-c566838aec17\") "
Oct 30 21:05:32.903975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.903819    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-data" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:05:32.904622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.904381    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:32.904622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.904445    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:32.904622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.904460    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:32.904622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.904476    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:32.905138 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.905103    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a626ea-7131-4ae2-a793-c566838aec17-istiod-ca-cert" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:05:32.906431 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.906342    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-kube-api-access-l5z98" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "kube-api-access-l5z98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:05:32.906431 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.906414    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-istio-token" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:05:32.906698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.906670    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-envoy" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:05:32.912011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:32.911988    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75a626ea-7131-4ae2-a793-c566838aec17-istio-podinfo" pod "75a626ea-7131-4ae2-a793-c566838aec17" (UID: "75a626ea-7131-4ae2-a793-c566838aec17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:05:33.006143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.006104    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5z98\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-kube-api-access-l5z98\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:33.006143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.006138    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a626ea-7131-4ae2-a793-c566838aec17-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:33.006143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.006149    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a626ea-7131-4ae2-a793-c566838aec17-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:33.006420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.006157    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a626ea-7131-4ae2-a793-c566838aec17-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:33.006420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.006166    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a626ea-7131-4ae2-a793-c566838aec17-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:33.329090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.329052    4852 generic.go:356] "Generic (PLEG): container finished" podID="75a626ea-7131-4ae2-a793-c566838aec17" containerID="21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d" exitCode=0
Oct 30 21:05:33.329668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.329101    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"
Oct 30 21:05:33.329668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.329129    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" event={"ID":"75a626ea-7131-4ae2-a793-c566838aec17","Type":"ContainerDied","Data":"21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d"}
Oct 30 21:05:33.329668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.329200    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2" event={"ID":"75a626ea-7131-4ae2-a793-c566838aec17","Type":"ContainerDied","Data":"ec3f1388a699b076e63963e1b3a9e0a66246fe465b782c41b3fc6c95149af703"}
Oct 30 21:05:33.329668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.329234    4852 scope.go:122] "RemoveContainer" containerID="21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d"
Oct 30 21:05:33.345475 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.345453    4852 scope.go:122] "RemoveContainer" containerID="21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d"
Oct 30 21:05:33.345826 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:05:33.345805    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d\": container with ID starting with 21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d not found: ID does not exist" containerID="21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d"
Oct 30 21:05:33.345888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.345835    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d"} err="failed to get container status \"21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d\": rpc error: code = NotFound desc = could not find container \"21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d\": container with ID starting with 21d143caa0de56f4b5becd7c446895230b3dfe8df655c58005a618ef7aa9975d not found: ID does not exist"
Oct 30 21:05:33.360191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.360157    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"]
Oct 30 21:05:33.365073 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.365045    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-l4bs2"]
Oct 30 21:05:33.562953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:33.562919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:34.006607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:34.006573    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75a626ea-7131-4ae2-a793-c566838aec17" path="/var/lib/kubelet/pods/75a626ea-7131-4ae2-a793-c566838aec17/volumes"
Oct 30 21:05:34.236739 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:34.236706    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:34.991508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:34.991475    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:35.863444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:35.863403    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:36.740450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:36.740377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:37.627600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:37.627565    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:38.433713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:38.433678    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:39.211177 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:39.211139    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:39.956505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:39.956470    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:40.753040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:40.753003    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:41.524269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:41.524235    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:42.344449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:42.344379    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:43.230786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:43.230740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:43.936639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:43.936599    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:44.639240 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:44.639179    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:45.411658 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:45.411628    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:46.262212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:46.262180    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:47.444057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:47.444019    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:48.515051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:48.515019    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:49.513243 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:49.513213    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:50.220534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:50.220498    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:50.939081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:50.939046    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:51.583889 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:05:51.583856    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:05:51.719844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:51.719812    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:52.427651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:52.427592    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:52.526673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:52.526589    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:53.113565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:53.113518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:53.873942 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:53.873906    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:54.658136 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:54.658098    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:55.416512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:55.416477    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:56.225970 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:56.225936    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:57.170122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:57.170082    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:57.906018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:57.905985    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:58.587982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.587947    4852 generic.go:356] "Generic (PLEG): container finished" podID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerID="096e63de71a8bfc2c1acd651e381fae1784565d91b64056c23e721155d93d204" exitCode=0
Oct 30 21:05:58.588215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.588144    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" event={"ID":"8fdcb224-4ee2-41da-942a-6298b75d1a0b","Type":"ContainerDied","Data":"096e63de71a8bfc2c1acd651e381fae1784565d91b64056c23e721155d93d204"}
Oct 30 21:05:58.589676 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.589649    4852 scope.go:122] "RemoveContainer" containerID="096e63de71a8bfc2c1acd651e381fae1784565d91b64056c23e721155d93d204"
Oct 30 21:05:58.591565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.591533    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:05:58.591686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.591612    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:05:58.591686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.591645    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:05:58.635513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.635482    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:58.877890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:58.877809    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:05:59.516555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:59.516518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:59.633623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:59.631770    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" event={"ID":"8fdcb224-4ee2-41da-942a-6298b75d1a0b","Type":"ContainerStarted","Data":"4180dab5ee7e4501af1a4ce2a349ba33aa7093387f4a7c34b3be3a6eed070a88"}
Oct 30 21:05:59.833494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:59.833359    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:05:59.834917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:59.834892    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:05:59.835051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:05:59.834940    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:06:00.229089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:00.229012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:00.838055 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:00.838020    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:06:00.838508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:00.838094    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:06:01.044403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:01.044365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:01.808706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:01.808666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:01.834083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:01.834054    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:06:01.834247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:01.834111    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:06:02.528238 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:02.528206    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:02.834183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:02.834095    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:06:02.834183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:02.834162    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:06:03.240813 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:03.240728    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:03.833891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:03.833855    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 21:06:03.834296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:03.833913    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 21:06:03.943333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:03.943303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:04.221852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:04.221775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:04.659068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:04.659025    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:04.879106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:04.879076    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:06:04.879708 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:04.879633    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:06:04.880568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:04.880547    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:06:05.439870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:05.439835    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:06.321682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:06.321649    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:07.160861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:07.160831    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:07.938982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:07.938943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:08.713651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:08.713616    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:09.414773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:09.414729    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:10.236063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:10.236023    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:10.582097 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:10.582064    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:10.683190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:10.683142    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:11.013313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:11.013285    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:11.166672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:11.166634    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:11.938612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:11.938581    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:12.679672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:12.679635    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:13.541004 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:13.540967    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:14.478736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:14.478699    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:15.455286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:15.455253    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:16.982500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:16.982463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:17.983891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:17.983850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:18.993630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:18.993582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:19.811440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:19.811411    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:20.657969 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:20.657926    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:20.761686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:20.761648    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:21.434535 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:21.434504    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:21.562787 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:06:21.562748    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:06:22.226849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:22.226806    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:23.012075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:23.011995    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:23.843099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:23.843059    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:24.536669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:24.536629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:25.304994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:25.304958    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:26.157277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:26.157247    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:26.923141 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:26.923048    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:27.637436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:27.637355    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:28.573045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:28.573012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:29.465761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:29.465714    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:30.234737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:30.234697    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:31.017419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:31.017352    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:31.739730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:31.739681    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:32.634279 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:32.634245    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:33.428269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:33.428241    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:34.316649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:34.316609    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:34.954246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:34.954195    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:35.134796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.134747    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:35.543064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.543025    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"]
Oct 30 21:06:35.544137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.544112    4852 cpu_manager.go:414] "containerMap: removing container" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy"
Oct 30 21:06:35.544235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.544140    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy"
Oct 30 21:06:35.544345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.544329    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="75a626ea-7131-4ae2-a793-c566838aec17" containerName="istio-proxy"
Oct 30 21:06:35.547437 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.547414    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.553900 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.553873    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-345bc4-openshift-default-dockercfg-fck6v\""
Oct 30 21:06:35.559793 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.559766    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"]
Oct 30 21:06:35.577637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.577606    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.577804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.577643    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.577804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.577741    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.577804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.577771    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.577986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.577937    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.577986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.577970    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcvqd\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-kube-api-access-rcvqd\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.578133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.578114    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.578200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.578160    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.578200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.578183    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.591960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.591919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:35.679608 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.679578    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.679608 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.679612    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.679842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.679663    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.679842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.679690    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.679842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.679813    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.679997 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.679847    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcvqd\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-kube-api-access-rcvqd\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.679990    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.680006    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680149 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.680068    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680149 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.680105    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680149 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.680143    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.680417    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680547 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.680511    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.680547 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.680526    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.682127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.682104    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.682258 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.682238    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.683189 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.683167    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:35.688890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.688869    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.689061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.689037    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcvqd\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-kube-api-access-rcvqd\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:35.786283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.786248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:35.872248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:35.872160    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:36.273842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:36.273809    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"]
Oct 30 21:06:36.274272 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:06:36.274242    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618802d8_a3bc_4e09_846a_b3fd0ca1327c.slice/crio-c98ead870e124dc48ad2be1f8b48360b01a6cafca1252ee01b9117a5260a87a1 WatchSource:0}: Error finding container c98ead870e124dc48ad2be1f8b48360b01a6cafca1252ee01b9117a5260a87a1: Status 404 returned error can't find the container with id c98ead870e124dc48ad2be1f8b48360b01a6cafca1252ee01b9117a5260a87a1
Oct 30 21:06:36.276978 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:36.276943    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:06:36.277080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:36.277011    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:06:36.277080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:36.277041    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:06:36.574319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:36.574234    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:36.987245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:36.987153    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" event={"ID":"618802d8-a3bc-4e09-846a-b3fd0ca1327c","Type":"ContainerStarted","Data":"075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e"}
Oct 30 21:06:36.987245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:36.987239    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" event={"ID":"618802d8-a3bc-4e09-846a-b3fd0ca1327c","Type":"ContainerStarted","Data":"c98ead870e124dc48ad2be1f8b48360b01a6cafca1252ee01b9117a5260a87a1"}
Oct 30 21:06:37.050425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:37.048785    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podStartSLOduration=2.04875995 podStartE2EDuration="2.04875995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:37.027716731 +0000 UTC m=+1705.142450922" watchObservedRunningTime="2026-10-30 21:06:37.04875995 +0000 UTC m=+1705.163494145"
Oct 30 21:06:37.873366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:37.873329    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:37.879563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:37.879535    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:37.905320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:37.905290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:37.995261 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:37.995233    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:37.996265 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:37.996243    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:06:38.939470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:38.939438    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:39.390931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:39.390896    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:39.490275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:39.490235    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:39.958125 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:39.958087    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:40.733538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:40.733508    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:40.857229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:40.857188    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:41.332564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:41.332528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:42.229665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:42.229625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:42.948837 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:42.948807    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:43.742530 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:43.742490    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:44.563459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:44.563423    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:45.517840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:45.517796    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:46.137428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:46.137369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:46.825859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:46.825828    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:47.744042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:47.743990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:48.436692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:48.436657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:49.235940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:49.235902    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:49.928184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:49.928152    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:50.639664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:50.639631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:51.452914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:51.451552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:51.559002 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:06:51.558963    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:06:52.296275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.296243    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"]
Oct 30 21:06:52.299583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.299563    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.308654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.308630    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-345bc4-serving-cert\""
Oct 30 21:06:52.315508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.315482    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"]
Oct 30 21:06:52.346707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.346678    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:52.389047 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.389018    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-metrics-certs\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.389222 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.389155    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjfgb\" (UniqueName: \"kubernetes.io/projected/83bce913-e437-4e71-8b9a-3a3ece1096c0-kube-api-access-tjfgb\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.389298 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.389263    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-maas-api-tls\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.389343 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.389295    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83bce913-e437-4e71-8b9a-3a3ece1096c0-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.460522 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.460478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:52.491180 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.491144    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-metrics-certs\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.491348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.491231    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjfgb\" (UniqueName: \"kubernetes.io/projected/83bce913-e437-4e71-8b9a-3a3ece1096c0-kube-api-access-tjfgb\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.491348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.491310    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-maas-api-tls\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.491348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.491331    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83bce913-e437-4e71-8b9a-3a3ece1096c0-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.492053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.492029    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83bce913-e437-4e71-8b9a-3a3ece1096c0-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.493889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.493868    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-metrics-certs\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.493967 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.493868    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-maas-api-tls\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.500592 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.500569    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjfgb\" (UniqueName: \"kubernetes.io/projected/83bce913-e437-4e71-8b9a-3a3ece1096c0-kube-api-access-tjfgb\") pod \"maas-api-e2e-extmod-a-345bc4-688689447c-rdswj\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.562729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.562633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:52.615455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.615418    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:52.918961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.918876    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"]
Oct 30 21:06:52.952368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:52.951742    4852 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:06:53.164881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:53.164844    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj" event={"ID":"83bce913-e437-4e71-8b9a-3a3ece1096c0","Type":"ContainerStarted","Data":"dd743d4c01c2316747b6df8a9920f90e11309c3d0b4189dd3b4684b40fa3239d"}
Oct 30 21:06:53.426848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:53.426796    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.127872 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:54.127831    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.175061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:54.175020    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj" event={"ID":"83bce913-e437-4e71-8b9a-3a3ece1096c0","Type":"ContainerStarted","Data":"58247c6b6fe93f8a6163521b37dd1809dde1d8ef30c8359d31a9e395e30b6e4d"}
Oct 30 21:06:54.175271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:54.175091    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:06:54.217007 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:54.216528    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj" podStartSLOduration=1.739336035 podStartE2EDuration="2.216502474s" totalImagesPullingTime="477.166439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:54.20027411 +0000 UTC m=+1722.315008325" watchObservedRunningTime="2026-10-30 21:06:54.216502474 +0000 UTC m=+1722.331236666"
Oct 30 21:06:54.832190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:54.832152    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:55.537316 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:55.537285    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:56.317989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:56.317956    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:57.059890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:57.059851    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:57.891483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:57.891453    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:58.875205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:06:58.875172    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:00.187981 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:00.187952    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:07:00.495692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:00.495610    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:00.607898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:00.607811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:01.317642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:01.317605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:02.036758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:02.036724    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:02.820423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:02.820377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:03.515688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:03.515658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:04.281447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:04.281382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:04.358133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:04.358076    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:05.220566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:05.220522    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:05.949862 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:05.949822    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:07.109120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:07.109086    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:08.246905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:08.246870    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:09.040652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:09.040606    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:09.845655 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:09.845614    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:10.246790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:10.246712    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:11.151175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:11.151143    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:11.252015 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:11.251979    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:11.621181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:11.621134    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:11.740910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:11.740874    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:12.223593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:12.223556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:12.932917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:12.932846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:13.723588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:13.723560    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:14.692578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:14.692546    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:15.220975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:15.220940    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:15.312789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:15.312755    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:15.672498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:15.672466    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:16.404350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:16.404313    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:16.506559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:16.506511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:16.974692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:16.974660    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:17.725660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:17.725624    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:18.423122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:18.423095    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:18.520308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:18.520269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:18.651474 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:18.651438    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:19.263344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:19.263308    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:19.365413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:19.365368    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:19.582492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:19.581171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:20.541817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:20.541778    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:21.408245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:21.408212    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:21.559416 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:07:21.559363    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:07:22.101513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:22.101475    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:22.840929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:22.840893    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:23.535378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:23.535337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:24.320476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:24.320432    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:25.119116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:25.119077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:25.334593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:25.334563    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:25.434677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:25.434583    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:26.080947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:26.080913    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:26.787253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:26.787221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:27.552999 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:27.552963    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:28.349943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:28.349909    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:28.446329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:28.446292    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:28.547985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:28.547871    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:29.497343 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:29.497256    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:30.322963 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:30.322931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:31.117564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:31.117529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:31.789353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:31.789313    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:31.890498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:31.890459    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:32.212312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:32.212234    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:32.830793 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:32.830764    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:33.549761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:33.549723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:34.342608 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:34.342577    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:35.150763 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.150724    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:35.173070 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.173033    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"]
Oct 30 21:07:35.177450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.177432    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.187697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.187673    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-89d993-openshift-default-dockercfg-tx9c6\""
Oct 30 21:07:35.194066 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.194036    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"]
Oct 30 21:07:35.270407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.258013    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-data\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.270826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.270794    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db044528-1d26-4c8b-afe8-b4409880c297-istio-podinfo\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.283410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.271047    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-istio-token\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.283707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.283675    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-envoy\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.283960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.283938    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92glp\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-kube-api-access-92glp\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.288783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.288740    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-certs\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.289664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.289643    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db044528-1d26-4c8b-afe8-b4409880c297-istiod-ca-cert\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.290006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.289987    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-socket\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.290213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.290196    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-credential-socket\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.290739 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.281787    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:35.391585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.391554    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-socket\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.391814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.391599    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-credential-socket\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.391814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.391672    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-data\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.391814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.391730    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db044528-1d26-4c8b-afe8-b4409880c297-istio-podinfo\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.391814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.391766    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-istio-token\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.391966    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-envoy\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392136 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392066    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-socket\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392136 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392081    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92glp\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-kube-api-access-92glp\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392136 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392116    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-credential-socket\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392150    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-certs\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392158    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-data\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392213    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db044528-1d26-4c8b-afe8-b4409880c297-istiod-ca-cert\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392408    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-certs\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.392810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.392785    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db044528-1d26-4c8b-afe8-b4409880c297-istiod-ca-cert\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.394395 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.394360    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-envoy\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.394700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.394680    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db044528-1d26-4c8b-afe8-b4409880c297-istio-podinfo\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.403354 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.403286    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-istio-token\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.410994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.410962    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92glp\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-kube-api-access-92glp\") pod \"e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") " pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.500162 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.500124    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:35.813899 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:07:35.813867    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb044528_1d26_4c8b_afe8_b4409880c297.slice/crio-56e55ce84dfb26d226eba2b2205d440085c23d90cd33d003da4c6ade37f431cc WatchSource:0}: Error finding container 56e55ce84dfb26d226eba2b2205d440085c23d90cd33d003da4c6ade37f431cc: Status 404 returned error can't find the container with id 56e55ce84dfb26d226eba2b2205d440085c23d90cd33d003da4c6ade37f431cc
Oct 30 21:07:35.814171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.814149    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"]
Oct 30 21:07:35.816680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.816645    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:07:35.816776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.816717    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:07:35.816776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.816750    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:07:35.841257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:35.841227    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:36.179814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:36.179709    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:36.298851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:36.297088    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:36.484249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:36.472740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:36.551103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:36.551059    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" event={"ID":"db044528-1d26-4c8b-afe8-b4409880c297","Type":"ContainerStarted","Data":"88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19"}
Oct 30 21:07:36.551321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:36.551121    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" event={"ID":"db044528-1d26-4c8b-afe8-b4409880c297","Type":"ContainerStarted","Data":"56e55ce84dfb26d226eba2b2205d440085c23d90cd33d003da4c6ade37f431cc"}
Oct 30 21:07:36.575249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:36.575194    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podStartSLOduration=1.575179592 podStartE2EDuration="1.575179592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:36.573122662 +0000 UTC m=+1764.687856847" watchObservedRunningTime="2026-10-30 21:07:36.575179592 +0000 UTC m=+1764.689913779"
Oct 30 21:07:37.125635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:37.125600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:37.501325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:37.501238    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:37.503070 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:37.503045    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 21:07:37.503206 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:37.503118    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 21:07:37.975410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:37.975251    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:38.072414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:38.071813    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:38.242147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:38.241310    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:38.501646 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:38.501568    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 21:07:38.501646 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:38.501631    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 21:07:38.575714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:38.575675    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:39.306995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:39.306961    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:39.501134 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:39.501095    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 21:07:39.501344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:39.501158    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 21:07:40.143455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:40.143345    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:40.381044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:40.381000    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:40.502374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:40.502286    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 21:07:40.502374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:40.502353    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 21:07:40.522634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:40.522600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:40.804437 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:40.798586    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:40.929764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:40.929702    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:41.011345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:41.011256    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:41.182876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:41.182795    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:41.500854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:41.500771    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 21:07:41.500854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:41.500833    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 21:07:41.594666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:41.594619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:42.066099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:42.066041    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:42.500639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:42.500552    4852 patch_prober.go:29] interesting pod/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 30 21:07:42.500639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:42.500619    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 30 21:07:42.969600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:42.969562    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:43.920812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:43.920769    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:43.992773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:43.992735    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:43.994568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:43.994540    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:44.012269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:44.012238    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:07:44.467993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:44.467956    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:44.682145 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:44.682092    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:44.951798 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:44.951761    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:45.708930 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:45.708888    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:45.861052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:45.861022    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:46.660945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:46.660908    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:46.802837 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:46.802802    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:47.626344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:47.626310    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:48.058566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:48.058528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:48.380458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:48.380355    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:49.238524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:49.238485    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:50.063323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:50.063269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:50.724676 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:50.724643    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:50.824182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:50.824141    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:50.939816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:50.939786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:51.181412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:51.160314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:51.375804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:51.375765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:51.559797 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:07:51.559765    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:07:51.567555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:51.567529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:51.796609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:51.796570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:51.897425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:51.897317    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.001882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:52.001842    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.176501 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:52.171037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.448688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:52.448612    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.707840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:52.707752    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.834995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:52.834955    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.931619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:52.931584    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:53.194443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:53.194403    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:53.291842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:53.291804    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:54.021411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:54.012942    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:54.212444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:54.212370    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:54.989004 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:54.988970    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:55.832587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:55.832554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:56.579409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:56.578820    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:56.672483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:56.672447    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:57.224644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:57.224606    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:58.116443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:58.115868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:59.743448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:07:59.743368    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:00.541465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:00.541398    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:00.643571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:00.643531    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:00.741831 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:00.741801    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:01.600835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:01.600799    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:02.362153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:02.362117    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:03.236411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:03.236301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:04.089164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:04.089119    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:04.326072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:04.326023    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:05.124418 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:05.124377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:05.933766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:05.933729    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:06.035661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:06.035621    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:06.838092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:06.838057    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:07.037664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:07.037635    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:07.137300 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:07.137218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:07.656563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:07.656522    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:08.446476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:08.446444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:09.264340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:09.264256    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:10.063639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:10.063574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:10.747667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:10.747632    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"]
Oct 30 21:08:10.747918 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:10.747892    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy" containerID="cri-o://0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba" gracePeriod=30
Oct 30 21:08:10.753507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:10.753467    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"]
Oct 30 21:08:10.753753 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:10.753726    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy" containerID="cri-o://7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f" gracePeriod=30
Oct 30 21:08:10.983029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:10.982992    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:11.756528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:11.756473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:12.216676 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:12.216589    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:12.772968 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:12.772916    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:12.878755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:12.878677    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:13.365984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:13.365948    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:13.636035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:13.635877    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:14.112307 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:14.112262    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:15.056093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:15.056060    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:15.910555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:15.910489    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:16.067179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.067141    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 21:08:16.144527 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.144495    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:08:16.199581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.199537    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-socket\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.199756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.199719    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-data\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.199866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.199851    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-token\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-token\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.199961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.199925    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-certs\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.200018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.199848    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-socket" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.200018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.199986    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istiod-ca-cert\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.200122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200013    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-data" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.200122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200077    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-podinfo\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.200122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200117    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-kube-api-access-qrnm4\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-kube-api-access-qrnm4\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.200275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200166    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-credential-socket\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.200275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200240    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-envoy\") pod \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\" (UID: \"d2c5027e-60b3-415f-a6d7-75e14d0a78f3\") "
Oct 30 21:08:16.200275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200250    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-certs" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.200505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200477    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-credential-socket" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.200698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.200674    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istiod-ca-cert" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:08:16.201048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.201027    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.201048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.201048    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.201192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.201059    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.201192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.201069    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.201192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.201078    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.202276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.202256    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-podinfo" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:08:16.202746 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.202727    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-envoy" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.202820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.202799    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-kube-api-access-qrnm4" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "kube-api-access-qrnm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:16.203053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.203040    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-token" pod "d2c5027e-60b3-415f-a6d7-75e14d0a78f3" (UID: "d2c5027e-60b3-415f-a6d7-75e14d0a78f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:16.301892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.301804    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-envoy\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.301892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.301849    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-certs\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.301892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.301888    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8118f70-aa12-47e3-8c81-03f576085993-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8118f70-aa12-47e3-8c81-03f576085993-istiod-ca-cert\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.302173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.301980    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-socket\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.302173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302036    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-credential-socket\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.302173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302076    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-istio-token\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-istio-token\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.302325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302161    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-certs" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.302325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302203    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-kube-api-access-stv8f\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-kube-api-access-stv8f\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.302455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302345    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-data\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.302455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302346    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-credential-socket" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.302455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302377    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b8118f70-aa12-47e3-8c81-03f576085993-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8118f70-aa12-47e3-8c81-03f576085993-istio-podinfo\") pod \"b8118f70-aa12-47e3-8c81-03f576085993\" (UID: \"b8118f70-aa12-47e3-8c81-03f576085993\") "
Oct 30 21:08:16.302611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302495    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-socket" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.302611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302528    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8118f70-aa12-47e3-8c81-03f576085993-istiod-ca-cert" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:08:16.302721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.302610    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-data" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.303083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303063    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303086    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303099    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrnm4\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-kube-api-access-qrnm4\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303112    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303125    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303139    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303153    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2c5027e-60b3-415f-a6d7-75e14d0a78f3-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303166    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.303194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.303177    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8118f70-aa12-47e3-8c81-03f576085993-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.304160 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.304131    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-envoy" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:16.304471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.304451    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-kube-api-access-stv8f" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "kube-api-access-stv8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:16.304668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.304649    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-istio-token" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:16.304745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.304729    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8118f70-aa12-47e3-8c81-03f576085993-istio-podinfo" pod "b8118f70-aa12-47e3-8c81-03f576085993" (UID: "b8118f70-aa12-47e3-8c81-03f576085993"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:08:16.404836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.404800    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stv8f\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-kube-api-access-stv8f\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.404836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.404832    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8118f70-aa12-47e3-8c81-03f576085993-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.404836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.404845    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8118f70-aa12-47e3-8c81-03f576085993-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.405081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.404854    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8118f70-aa12-47e3-8c81-03f576085993-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:16.561309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.561216    4852 generic.go:356] "Generic (PLEG): container finished" podID="b8118f70-aa12-47e3-8c81-03f576085993" containerID="7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f" exitCode=0
Oct 30 21:08:16.561309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.561271    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"
Oct 30 21:08:16.561309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.561299    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" event={"ID":"b8118f70-aa12-47e3-8c81-03f576085993","Type":"ContainerDied","Data":"7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f"}
Oct 30 21:08:16.561635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.561380    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r" event={"ID":"b8118f70-aa12-47e3-8c81-03f576085993","Type":"ContainerDied","Data":"9c3b0fd741b24fdf1cbccfa1072254e4848f09dbafc62331236276ba25e48eaf"}
Oct 30 21:08:16.561635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.561423    4852 scope.go:122] "RemoveContainer" containerID="7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f"
Oct 30 21:08:16.563154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.563129    4852 generic.go:356] "Generic (PLEG): container finished" podID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerID="0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba" exitCode=0
Oct 30 21:08:16.563300 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.563160    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" event={"ID":"d2c5027e-60b3-415f-a6d7-75e14d0a78f3","Type":"ContainerDied","Data":"0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba"}
Oct 30 21:08:16.563300 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.563205    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"
Oct 30 21:08:16.563300 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.563209    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn" event={"ID":"d2c5027e-60b3-415f-a6d7-75e14d0a78f3","Type":"ContainerDied","Data":"1b65743dff67eb0f983132535d3c1637a5d641057a821f8d83763989b4ec1a64"}
Oct 30 21:08:16.574727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.574702    4852 scope.go:122] "RemoveContainer" containerID="7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f"
Oct 30 21:08:16.575017 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:16.574996    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f\": container with ID starting with 7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f not found: ID does not exist" containerID="7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f"
Oct 30 21:08:16.575104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.575028    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f"} err="failed to get container status \"7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f\": rpc error: code = NotFound desc = could not find container \"7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f\": container with ID starting with 7a530bd4bb21d97c2aaa114ef652cdd0aa9be4b7bdd9af142b3e7ad7760cce3f not found: ID does not exist"
Oct 30 21:08:16.575104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.575051    4852 scope.go:122] "RemoveContainer" containerID="0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba"
Oct 30 21:08:16.590924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.590899    4852 scope.go:122] "RemoveContainer" containerID="0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba"
Oct 30 21:08:16.591265 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:16.591240    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba\": container with ID starting with 0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba not found: ID does not exist" containerID="0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba"
Oct 30 21:08:16.591349 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.591274    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba"} err="failed to get container status \"0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba\": rpc error: code = NotFound desc = could not find container \"0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba\": container with ID starting with 0d876cd37d92df8b8ae9563a1117cb8d2a419194ca7d9dac190b12be99e46fba not found: ID does not exist"
Oct 30 21:08:16.595303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.595276    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"]
Oct 30 21:08:16.600026 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.600002    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-ncjdn"]
Oct 30 21:08:16.611108 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.611085    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"]
Oct 30 21:08:16.614904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.614878    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-4bq6r"]
Oct 30 21:08:16.890226 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:16.890102    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:17.341885 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:17.341848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:17.447188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:17.447153    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:17.990577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:17.990543    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:18.007421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:18.007365    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8118f70-aa12-47e3-8c81-03f576085993" path="/var/lib/kubelet/pods/b8118f70-aa12-47e3-8c81-03f576085993/volumes"
Oct 30 21:08:18.007961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:18.007945    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" path="/var/lib/kubelet/pods/d2c5027e-60b3-415f-a6d7-75e14d0a78f3/volumes"
Oct 30 21:08:18.545924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:18.545889    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:18.647204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:18.647172    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:19.285139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:19.285100    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:19.964453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:19.964421    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:20.346187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:20.346158    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:21.146299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:21.146267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:21.340641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:21.340602    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:21.559982 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:21.559943    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:08:21.825151 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:21.825065    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:22.175355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:22.175272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:22.372337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:22.372301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:23.528179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:23.528146    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:23.892008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:23.891907    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:24.025672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:24.025638    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"]
Oct 30 21:08:24.025870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:24.025844    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" containerID="cri-o://af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5" gracePeriod=30
Oct 30 21:08:24.348315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:24.348281    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:25.200078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:25.200046    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:25.356785 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:25.356741    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:25.462491 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:25.462369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:26.396770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:26.396736    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:26.662274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:26.662189    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:26.767717 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:26.767682    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:27.515255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:27.515218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:27.837356 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:27.837275    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-2h8ll container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:08:27.837356 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:27.837335    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:08:28.230318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:28.230244    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:29.296471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.296421    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:29.333839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.333769    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 21:08:29.462779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.462745    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-socket\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.462951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.462901    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-data\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463014 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.462964    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istiod-ca-cert\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463049    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-credential-socket\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463048    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-socket" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:29.463182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463088    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-envoy\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463117    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-certs\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463157    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-data" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:29.463182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463176    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-podinfo\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463437 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463199    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-token\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-token\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463437 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463232    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-kube-api-access-z5ht7\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-kube-api-access-z5ht7\") pod \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\" (UID: \"64c9e9e4-3f97-4165-b06e-b15c24cd9eba\") "
Oct 30 21:08:29.463437 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463291    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-credential-socket" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:29.463595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463493    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-certs" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:29.463595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463523    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istiod-ca-cert" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:08:29.464008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.463980    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.464106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.464010    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.464106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.464029    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.464106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.464045    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.464106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.464059    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.465378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.465353    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-envoy" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:29.465496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.465404    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-podinfo" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:08:29.465543 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.465498    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-kube-api-access-z5ht7" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "kube-api-access-z5ht7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:29.465961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.465936    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-token" pod "64c9e9e4-3f97-4165-b06e-b15c24cd9eba" (UID: "64c9e9e4-3f97-4165-b06e-b15c24cd9eba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:29.565921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.565880    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.565921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.565924    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.566202 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.565941    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.566202 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.565956    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5ht7\" (UniqueName: \"kubernetes.io/projected/64c9e9e4-3f97-4165-b06e-b15c24cd9eba-kube-api-access-z5ht7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:29.720255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.720156    4852 generic.go:356] "Generic (PLEG): container finished" podID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerID="af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5" exitCode=0
Oct 30 21:08:29.720255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.720232    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"
Oct 30 21:08:29.720255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.720241    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" event={"ID":"64c9e9e4-3f97-4165-b06e-b15c24cd9eba","Type":"ContainerDied","Data":"af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5"}
Oct 30 21:08:29.720572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.720324    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll" event={"ID":"64c9e9e4-3f97-4165-b06e-b15c24cd9eba","Type":"ContainerDied","Data":"dd9e851293815f583b490c16daf7d0eac7a427484c09568fcd417c527dc647d4"}
Oct 30 21:08:29.720572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.720348    4852 scope.go:122] "RemoveContainer" containerID="af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5"
Oct 30 21:08:29.736079 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.736057    4852 scope.go:122] "RemoveContainer" containerID="f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4"
Oct 30 21:08:29.747627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.747603    4852 scope.go:122] "RemoveContainer" containerID="af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5"
Oct 30 21:08:29.748106 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:29.748067    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5\": container with ID starting with af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5 not found: ID does not exist" containerID="af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5"
Oct 30 21:08:29.748224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.748108    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5"} err="failed to get container status \"af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5\": rpc error: code = NotFound desc = could not find container \"af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5\": container with ID starting with af3070da5dcd945f84da694661c615be48c0de840d8aaa3df231d25100285bf5 not found: ID does not exist"
Oct 30 21:08:29.748224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.748131    4852 scope.go:122] "RemoveContainer" containerID="f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4"
Oct 30 21:08:29.748511 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:29.748477    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4\": container with ID starting with f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4 not found: ID does not exist" containerID="f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4"
Oct 30 21:08:29.748630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.748515    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4"} err="failed to get container status \"f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4\": rpc error: code = NotFound desc = could not find container \"f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4\": container with ID starting with f87f8bd3c47b79ca252abfe715ae015a396807f1a35a2b6de3b9d67a79db60a4 not found: ID does not exist"
Oct 30 21:08:29.750021 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.750004    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"]
Oct 30 21:08:29.753922 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.753901    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-2h8ll"]
Oct 30 21:08:29.791890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.791862    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"]
Oct 30 21:08:29.792084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:29.792064    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy" containerID="cri-o://481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3" gracePeriod=30
Oct 30 21:08:30.006790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:30.006762    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" path="/var/lib/kubelet/pods/64c9e9e4-3f97-4165-b06e-b15c24cd9eba/volumes"
Oct 30 21:08:30.446379 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:30.446303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:31.546299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:31.546241    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:32.512765 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:32.512725    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:33.544285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:33.544249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:33.746297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:33.746242    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:33.848772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:33.848663    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:34.088655 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:34.088618    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"]
Oct 30 21:08:34.088863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:34.088840    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy" containerID="cri-o://88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19" gracePeriod=30
Oct 30 21:08:34.139934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:34.139856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:34.776548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:34.776512    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:34.917169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:34.917140    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:35.109605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.109575    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:08:35.267660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.267625    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2aa6872-a8ec-463e-8524-040ce926883d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2aa6872-a8ec-463e-8524-040ce926883d-istiod-ca-cert\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.267865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.267681    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-data\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.267865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.267724    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-socket\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.267992 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.267871    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-istio-token\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-istio-token\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.267992 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.267929    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-envoy\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.268100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268049    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-kube-api-access-xnjnp\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-kube-api-access-xnjnp\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.268100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268052    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-data" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:35.268210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268120    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2aa6872-a8ec-463e-8524-040ce926883d-istiod-ca-cert" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:08:35.268210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268143    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c2aa6872-a8ec-463e-8524-040ce926883d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2aa6872-a8ec-463e-8524-040ce926883d-istio-podinfo\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.268210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268184    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-credential-socket\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.268369 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268239    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-certs\") pod \"c2aa6872-a8ec-463e-8524-040ce926883d\" (UID: \"c2aa6872-a8ec-463e-8524-040ce926883d\") "
Oct 30 21:08:35.268369 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268265    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-socket" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:35.268630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268605    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-credential-socket" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:35.268814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268720    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-certs" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:35.268950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268931    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.269020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268959    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.269020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268974    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2aa6872-a8ec-463e-8524-040ce926883d-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.269020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.268988    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.269020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.269003    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.270638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.270616    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-kube-api-access-xnjnp" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "kube-api-access-xnjnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:35.270865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.270846    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2aa6872-a8ec-463e-8524-040ce926883d-istio-podinfo" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:08:35.270924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.270880    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-envoy" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:35.271759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.271738    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-istio-token" pod "c2aa6872-a8ec-463e-8524-040ce926883d" (UID: "c2aa6872-a8ec-463e-8524-040ce926883d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:35.370983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.370911    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnjnp\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-kube-api-access-xnjnp\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.370983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.370940    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2aa6872-a8ec-463e-8524-040ce926883d-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.370983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.370950    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2aa6872-a8ec-463e-8524-040ce926883d-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.370983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.370960    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2aa6872-a8ec-463e-8524-040ce926883d-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:35.772552 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.772517    4852 generic.go:356] "Generic (PLEG): container finished" podID="c2aa6872-a8ec-463e-8524-040ce926883d" containerID="481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3" exitCode=0
Oct 30 21:08:35.772853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.772608    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"
Oct 30 21:08:35.773002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.772607    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" event={"ID":"c2aa6872-a8ec-463e-8524-040ce926883d","Type":"ContainerDied","Data":"481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3"}
Oct 30 21:08:35.773069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.773016    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6" event={"ID":"c2aa6872-a8ec-463e-8524-040ce926883d","Type":"ContainerDied","Data":"cf3b5bc3eb8090790baacfd37cb7cd5a5c5b94c78ac4da442d9614a9480f16d8"}
Oct 30 21:08:35.773069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.773058    4852 scope.go:122] "RemoveContainer" containerID="481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3"
Oct 30 21:08:35.788253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.788173    4852 scope.go:122] "RemoveContainer" containerID="481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3"
Oct 30 21:08:35.788671 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:35.788640    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3\": container with ID starting with 481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3 not found: ID does not exist" containerID="481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3"
Oct 30 21:08:35.788745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.788680    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3"} err="failed to get container status \"481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3\": rpc error: code = NotFound desc = could not find container \"481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3\": container with ID starting with 481fb4639b21efdf5bd67a923f5c8e2cfc1922930e4e7e6324db482fd5c8b4b3 not found: ID does not exist"
Oct 30 21:08:35.804341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.804310    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"]
Oct 30 21:08:35.810905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.810874    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-x2xs6"]
Oct 30 21:08:35.985338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.985302    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"]
Oct 30 21:08:35.986248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986229    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy"
Oct 30 21:08:35.986296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986251    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy"
Oct 30 21:08:35.986296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986271    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy"
Oct 30 21:08:35.986296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986278    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy"
Oct 30 21:08:35.986296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986288    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy"
Oct 30 21:08:35.986296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986295    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy"
Oct 30 21:08:35.986471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986322    4852 cpu_manager.go:414] "containerMap: removing container" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy"
Oct 30 21:08:35.986471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986328    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy"
Oct 30 21:08:35.986471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986337    4852 cpu_manager.go:414] "containerMap: removing container" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy"
Oct 30 21:08:35.986471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986342    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy"
Oct 30 21:08:35.986471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986456    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy"
Oct 30 21:08:35.986471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986468    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="64c9e9e4-3f97-4165-b06e-b15c24cd9eba" containerName="istio-proxy"
Oct 30 21:08:35.986662 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986478    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" containerName="istio-proxy"
Oct 30 21:08:35.986662 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986487    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8118f70-aa12-47e3-8c81-03f576085993" containerName="istio-proxy"
Oct 30 21:08:35.986662 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.986492    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2c5027e-60b3-415f-a6d7-75e14d0a78f3" containerName="istio-proxy"
Oct 30 21:08:35.991653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:35.991627    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.004039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.003542    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"]
Oct 30 21:08:36.019911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.019877    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2aa6872-a8ec-463e-8524-040ce926883d" path="/var/lib/kubelet/pods/c2aa6872-a8ec-463e-8524-040ce926883d/volumes"
Oct 30 21:08:36.052133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.052030    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:36.053600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.053584    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-435414-serving-cert\""
Oct 30 21:08:36.082184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.082149    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-metrics-certs\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.082363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.082206    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qdgz\" (UniqueName: \"kubernetes.io/projected/aed980dc-f631-427f-aaa9-af833637e796-kube-api-access-7qdgz\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.082363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.082330    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-maas-api-tls\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.082502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.082395    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed980dc-f631-427f-aaa9-af833637e796-openshift-service-ca\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.153641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.153611    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:36.187571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.187530    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-metrics-certs\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.187836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.187591    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qdgz\" (UniqueName: \"kubernetes.io/projected/aed980dc-f631-427f-aaa9-af833637e796-kube-api-access-7qdgz\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.187836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.187662    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-maas-api-tls\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.187836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.187719    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed980dc-f631-427f-aaa9-af833637e796-openshift-service-ca\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.190482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.189886    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed980dc-f631-427f-aaa9-af833637e796-openshift-service-ca\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.195083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.193099    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-metrics-certs\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.198751 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.198717    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qdgz\" (UniqueName: \"kubernetes.io/projected/aed980dc-f631-427f-aaa9-af833637e796-kube-api-access-7qdgz\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.199894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.199838    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-maas-api-tls\") pod \"maas-api-e2e-autores-435414-895f5d99c-ng5px\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.253479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.253438    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:36.326902 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.326816    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:36.611424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.611378    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"]
Oct 30 21:08:36.611546 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:08:36.611436    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed980dc_f631_427f_aaa9_af833637e796.slice/crio-21d2567087fc4c08281fd1fc12739decfdc525fc3224b36b53f175354dd0f716 WatchSource:0}: Error finding container 21d2567087fc4c08281fd1fc12739decfdc525fc3224b36b53f175354dd0f716: Status 404 returned error can't find the container with id 21d2567087fc4c08281fd1fc12739decfdc525fc3224b36b53f175354dd0f716
Oct 30 21:08:36.692172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.692135    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:36.781550 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.781513    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px" event={"ID":"aed980dc-f631-427f-aaa9-af833637e796","Type":"ContainerStarted","Data":"21d2567087fc4c08281fd1fc12739decfdc525fc3224b36b53f175354dd0f716"}
Oct 30 21:08:36.792344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:36.792316    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:37.222925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:37.222868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:37.791771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:37.791739    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px" event={"ID":"aed980dc-f631-427f-aaa9-af833637e796","Type":"ContainerStarted","Data":"e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65"}
Oct 30 21:08:37.791988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:37.791810    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:37.810565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:37.810515    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px" podStartSLOduration=2.343249515 podStartE2EDuration="2.810500854s" totalImagesPullingTime="467.251339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:37.808227552 +0000 UTC m=+1825.922961742" watchObservedRunningTime="2026-10-30 21:08:37.810500854 +0000 UTC m=+1825.925235043"
Oct 30 21:08:38.226795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:38.226717    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:39.055073 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.055033    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:39.404158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.404129    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:08:39.444927 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.444895    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-certs\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.444991    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-socket\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445028    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-credential-socket\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445099    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-envoy\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445166    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-kube-api-access-92glp\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-kube-api-access-92glp\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445210    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-certs" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:39.445242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445225    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-istio-token\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-istio-token\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445319    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-socket" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:39.445515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445339    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-data\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445433    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db044528-1d26-4c8b-afe8-b4409880c297-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db044528-1d26-4c8b-afe8-b4409880c297-istiod-ca-cert\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.445515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.445470    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db044528-1d26-4c8b-afe8-b4409880c297-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db044528-1d26-4c8b-afe8-b4409880c297-istio-podinfo\") pod \"db044528-1d26-4c8b-afe8-b4409880c297\" (UID: \"db044528-1d26-4c8b-afe8-b4409880c297\") "
Oct 30 21:08:39.446051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.446030    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.446140 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.446057    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.446140 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.446070    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-credential-socket" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:39.446415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.446152    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-data" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:39.446415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.446306    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db044528-1d26-4c8b-afe8-b4409880c297-istiod-ca-cert" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:08:39.447943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.447921    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-istio-token" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:39.448717 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.448695    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-envoy" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:08:39.449233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.449200    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-kube-api-access-92glp" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "kube-api-access-92glp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:39.449325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.449282    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db044528-1d26-4c8b-afe8-b4409880c297-istio-podinfo" pod "db044528-1d26-4c8b-afe8-b4409880c297" (UID: "db044528-1d26-4c8b-afe8-b4409880c297"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:08:39.547157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.547120    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.547157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.547151    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.547157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.547161    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92glp\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-kube-api-access-92glp\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.547433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.547171    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db044528-1d26-4c8b-afe8-b4409880c297-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.547433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.547181    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db044528-1d26-4c8b-afe8-b4409880c297-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.547433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.547190    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db044528-1d26-4c8b-afe8-b4409880c297-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.547433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.547200    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db044528-1d26-4c8b-afe8-b4409880c297-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:39.808988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.808955    4852 generic.go:356] "Generic (PLEG): container finished" podID="db044528-1d26-4c8b-afe8-b4409880c297" containerID="88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19" exitCode=0
Oct 30 21:08:39.809187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.809021    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"
Oct 30 21:08:39.809187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.809038    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" event={"ID":"db044528-1d26-4c8b-afe8-b4409880c297","Type":"ContainerDied","Data":"88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19"}
Oct 30 21:08:39.809187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.809091    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9" event={"ID":"db044528-1d26-4c8b-afe8-b4409880c297","Type":"ContainerDied","Data":"56e55ce84dfb26d226eba2b2205d440085c23d90cd33d003da4c6ade37f431cc"}
Oct 30 21:08:39.809187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.809114    4852 scope.go:122] "RemoveContainer" containerID="88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19"
Oct 30 21:08:39.822435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.822353    4852 scope.go:122] "RemoveContainer" containerID="88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19"
Oct 30 21:08:39.822862 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:39.822837    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19\": container with ID starting with 88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19 not found: ID does not exist" containerID="88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19"
Oct 30 21:08:39.822988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.822872    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19"} err="failed to get container status \"88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19\": rpc error: code = NotFound desc = could not find container \"88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19\": container with ID starting with 88e5e59467be0dcdc5fac0928662ab1c568948e7d398e53997d276848dec4d19 not found: ID does not exist"
Oct 30 21:08:39.835854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.835827    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"]
Oct 30 21:08:39.844154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:39.844130    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-89d993-openshift-default-665f5486f7-p8jx9"]
Oct 30 21:08:40.005898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:40.005866    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db044528-1d26-4c8b-afe8-b4409880c297" path="/var/lib/kubelet/pods/db044528-1d26-4c8b-afe8-b4409880c297/volumes"
Oct 30 21:08:40.190917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:40.190830    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:40.645605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:40.645575    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:40.745846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:40.745815    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:41.053061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.053030    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:41.860071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.860037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:41.960492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.960461    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:41.977151 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.977123    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"]
Oct 30 21:08:41.977816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.977801    4852 cpu_manager.go:414] "containerMap: removing container" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy"
Oct 30 21:08:41.977868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.977817    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy"
Oct 30 21:08:41.977939 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.977930    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="db044528-1d26-4c8b-afe8-b4409880c297" containerName="istio-proxy"
Oct 30 21:08:41.983321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.983298    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:41.990481 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.990453    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-6741b1\""
Oct 30 21:08:41.990628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.990530    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-6741b1-dockercfg-kpw85\""
Oct 30 21:08:41.991612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:41.991588    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"]
Oct 30 21:08:42.020331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.020301    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"]
Oct 30 21:08:42.024598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.024572    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.041961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.041930    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"]
Oct 30 21:08:42.240953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.240846    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-272g4\" (UniqueName: \"kubernetes.io/projected/ba15be70-b3ba-49d5-b8fd-d4333a112656-kube-api-access-272g4\") pod \"payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:42.240953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.240921    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.241208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.241042    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5gts\" (UniqueName: \"kubernetes.io/projected/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-kube-api-access-t5gts\") pod \"payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.241329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.241311    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba15be70-b3ba-49d5-b8fd-d4333a112656-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:42.342860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.342826    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba15be70-b3ba-49d5-b8fd-d4333a112656-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:42.343046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.342944    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-272g4\" (UniqueName: \"kubernetes.io/projected/ba15be70-b3ba-49d5-b8fd-d4333a112656-kube-api-access-272g4\") pod \"payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:42.343046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.342980    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.343046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.343006    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5gts\" (UniqueName: \"kubernetes.io/projected/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-kube-api-access-t5gts\") pod \"payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.343721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.343694    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba15be70-b3ba-49d5-b8fd-d4333a112656-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:42.343830 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.343723    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.351477 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.351457    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5gts\" (UniqueName: \"kubernetes.io/projected/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-kube-api-access-t5gts\") pod \"payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.351572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.351482    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-272g4\" (UniqueName: \"kubernetes.io/projected/ba15be70-b3ba-49d5-b8fd-d4333a112656-kube-api-access-272g4\") pod \"payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:42.600443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.600407    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:42.643657 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.643618    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:42.811556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.811525    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:42.932958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.932927    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"]
Oct 30 21:08:42.982765 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:42.982360    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"]
Oct 30 21:08:42.982765 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:08:42.982712    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50dbd7f_3da6_40fa_b0d6_0f18db33671e.slice/crio-df22b59aca75a31c87c6fb973a424855640dc6dd9845cc350c7621b203ed43d2 WatchSource:0}: Error finding container df22b59aca75a31c87c6fb973a424855640dc6dd9845cc350c7621b203ed43d2: Status 404 returned error can't find the container with id df22b59aca75a31c87c6fb973a424855640dc6dd9845cc350c7621b203ed43d2
Oct 30 21:08:43.754020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.753988    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:43.805911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.805882    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:08:43.872146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.871897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:43.889353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.889319    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" event={"ID":"ba15be70-b3ba-49d5-b8fd-d4333a112656","Type":"ContainerStarted","Data":"27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86"}
Oct 30 21:08:43.889644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.889382    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" event={"ID":"ba15be70-b3ba-49d5-b8fd-d4333a112656","Type":"ContainerStarted","Data":"37171da7222286ad004e395756f496d5c548e00e6aeac281db41e9eba3c2dbd3"}
Oct 30 21:08:43.890918 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.890668    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:43.906928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.906893    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" event={"ID":"a50dbd7f-3da6-40fa-b0d6-0f18db33671e","Type":"ContainerStarted","Data":"c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7"}
Oct 30 21:08:43.907132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.906956    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" event={"ID":"a50dbd7f-3da6-40fa-b0d6-0f18db33671e","Type":"ContainerStarted","Data":"df22b59aca75a31c87c6fb973a424855640dc6dd9845cc350c7621b203ed43d2"}
Oct 30 21:08:43.907286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.907268    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:43.912688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.912636    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" podStartSLOduration=2.912610759 podStartE2EDuration="2.912610759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:43.90968952 +0000 UTC m=+1832.024423722" watchObservedRunningTime="2026-10-30 21:08:43.912610759 +0000 UTC m=+1832.027344948"
Oct 30 21:08:43.947417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.947347    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" podStartSLOduration=2.947330386 podStartE2EDuration="2.947330386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:43.945307074 +0000 UTC m=+1832.060041264" watchObservedRunningTime="2026-10-30 21:08:43.947330386 +0000 UTC m=+1832.062064576"
Oct 30 21:08:43.993365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:43.993335    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:44.957747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:44.957711    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:44.995639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:44.995607    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"]
Oct 30 21:08:45.061193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:45.060908    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:45.159644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:45.159601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:46.044485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:46.044445    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:46.883854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:46.883784    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:47.639623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:47.639589    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:48.426970 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:48.426931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:49.141318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:49.141286    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:49.843716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:49.843682    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:49.996072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:49.996027    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l" podUID="f4097963-cd6c-416a-b98d-83bcb875d7b2" containerName="payload-processing" containerID="cri-o://a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd" gracePeriod=25
Oct 30 21:08:50.278828 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.278795    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:08:50.375411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.375266    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4097963-cd6c-416a-b98d-83bcb875d7b2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4097963-cd6c-416a-b98d-83bcb875d7b2-plugins-config-volume\") pod \"f4097963-cd6c-416a-b98d-83bcb875d7b2\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") "
Oct 30 21:08:50.375411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.375357    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4097963-cd6c-416a-b98d-83bcb875d7b2-kube-api-access-r4z4p\" (UniqueName: \"kubernetes.io/projected/f4097963-cd6c-416a-b98d-83bcb875d7b2-kube-api-access-r4z4p\") pod \"f4097963-cd6c-416a-b98d-83bcb875d7b2\" (UID: \"f4097963-cd6c-416a-b98d-83bcb875d7b2\") "
Oct 30 21:08:50.376032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.376003    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4097963-cd6c-416a-b98d-83bcb875d7b2-plugins-config-volume" pod "f4097963-cd6c-416a-b98d-83bcb875d7b2" (UID: "f4097963-cd6c-416a-b98d-83bcb875d7b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:08:50.376203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.376183    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4097963-cd6c-416a-b98d-83bcb875d7b2-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:50.378023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.378001    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4097963-cd6c-416a-b98d-83bcb875d7b2-kube-api-access-r4z4p" pod "f4097963-cd6c-416a-b98d-83bcb875d7b2" (UID: "f4097963-cd6c-416a-b98d-83bcb875d7b2"). InnerVolumeSpecName "kube-api-access-r4z4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:50.477700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.477666    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4z4p\" (UniqueName: \"kubernetes.io/projected/f4097963-cd6c-416a-b98d-83bcb875d7b2-kube-api-access-r4z4p\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:50.613400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.613358    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:50.964713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.964679    4852 generic.go:356] "Generic (PLEG): container finished" podID="f4097963-cd6c-416a-b98d-83bcb875d7b2" containerID="a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd" exitCode=0
Oct 30 21:08:50.964895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.964727    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"
Oct 30 21:08:50.964895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.964743    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l" event={"ID":"f4097963-cd6c-416a-b98d-83bcb875d7b2","Type":"ContainerDied","Data":"a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd"}
Oct 30 21:08:50.964895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.964786    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l" event={"ID":"f4097963-cd6c-416a-b98d-83bcb875d7b2","Type":"ContainerDied","Data":"1e4d80c04b51f8caced66750de857e5e82fc748f202a348fdc26db09ea06dec1"}
Oct 30 21:08:50.964895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.964808    4852 scope.go:122] "RemoveContainer" containerID="a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd"
Oct 30 21:08:50.975983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.975964    4852 scope.go:122] "RemoveContainer" containerID="a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd"
Oct 30 21:08:50.976280 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:50.976259    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd\": container with ID starting with a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd not found: ID does not exist" containerID="a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd"
Oct 30 21:08:50.976342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.976289    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd"} err="failed to get container status \"a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd\": rpc error: code = NotFound desc = could not find container \"a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd\": container with ID starting with a2afea1696ac419fc85d699518d829e28bc3eb4f82d5bcb1187a3e7db32f18cd not found: ID does not exist"
Oct 30 21:08:50.991124 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.991096    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"]
Oct 30 21:08:50.995411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:50.995356    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-49c7c2-6f8cd548bf-mqw2l"]
Oct 30 21:08:51.311110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:51.311066    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:51.559430 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:08:51.559380    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:08:52.006143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:52.006109    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4097963-cd6c-416a-b98d-83bcb875d7b2" path="/var/lib/kubelet/pods/f4097963-cd6c-416a-b98d-83bcb875d7b2/volumes"
Oct 30 21:08:52.092723 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:52.092678    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:52.918024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:52.917990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:53.735689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:53.735653    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:54.609562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:54.609525    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:54.918053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:54.917976    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:08:55.304284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:55.304250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:55.474258 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:55.474220    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:55.573581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:55.573477    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:55.680472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:55.675375    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:55.925137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:55.925054    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:08:56.141357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.141328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:56.584967 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.584934    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"]
Oct 30 21:08:56.586021 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.586001    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f4097963-cd6c-416a-b98d-83bcb875d7b2" containerName="payload-processing"
Oct 30 21:08:56.586087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.586024    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4097963-cd6c-416a-b98d-83bcb875d7b2" containerName="payload-processing"
Oct 30 21:08:56.586232 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.586219    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4097963-cd6c-416a-b98d-83bcb875d7b2" containerName="payload-processing"
Oct 30 21:08:56.590498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.590473    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.598667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.598630    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-6741b1-serving-cert\""
Oct 30 21:08:56.598667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.598644    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"]
Oct 30 21:08:56.638342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.638309    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:56.713649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.713612    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.714029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.713673    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh8pq\" (UniqueName: \"kubernetes.io/projected/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-kube-api-access-jh8pq\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.714029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.713785    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.714029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.713831    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-metrics-certs\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.734664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.734627    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:56.816744 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.816712    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.817070 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.817045    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh8pq\" (UniqueName: \"kubernetes.io/projected/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-kube-api-access-jh8pq\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.817282 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.817264    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.817767 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.817743    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-metrics-certs\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.819539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.819516    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.823597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.823560    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-metrics-certs\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.829863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.827834    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.833034 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.832980    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh8pq\" (UniqueName: \"kubernetes.io/projected/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-kube-api-access-jh8pq\") pod \"maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:56.837555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.837486    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:56.919453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:56.919422    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:57.213714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:57.213681    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"]
Oct 30 21:08:57.214023 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:08:57.213999    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4af4c2_7566_4a42_aa50_5ca11beaa48c.slice/crio-e7a0af569b75d0e0a664decd0bc487e2cb0ada576ab6f4e272d4234527ce9127 WatchSource:0}: Error finding container e7a0af569b75d0e0a664decd0bc487e2cb0ada576ab6f4e272d4234527ce9127: Status 404 returned error can't find the container with id e7a0af569b75d0e0a664decd0bc487e2cb0ada576ab6f4e272d4234527ce9127
Oct 30 21:08:57.601955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:57.601919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:57.920422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:57.919450    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:58.035272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:58.035231    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z" event={"ID":"4b4af4c2-7566-4a42-aa50-5ca11beaa48c","Type":"ContainerStarted","Data":"8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6"}
Oct 30 21:08:58.035459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:58.035297    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z" event={"ID":"4b4af4c2-7566-4a42-aa50-5ca11beaa48c","Type":"ContainerStarted","Data":"e7a0af569b75d0e0a664decd0bc487e2cb0ada576ab6f4e272d4234527ce9127"}
Oct 30 21:08:58.035535 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:58.035467    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:08:58.054879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:58.054829    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z" podStartSLOduration=1.5243768599999998 podStartE2EDuration="2.054813801s" totalImagesPullingTime="530.436941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:58.053145977 +0000 UTC m=+1846.167880168" watchObservedRunningTime="2026-10-30 21:08:58.054813801 +0000 UTC m=+1846.169547991"
Oct 30 21:08:58.732132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:58.732093    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:59.588874 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:59.588832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:59.668100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:08:59.668062    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:00.694808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:00.694779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:01.533300 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:01.533267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:02.599112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:02.599046    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:02.701471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:02.701403    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:02.936654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:02.936612    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"]
Oct 30 21:09:02.937554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:02.937512    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy" containerID="cri-o://4180dab5ee7e4501af1a4ce2a349ba33aa7093387f4a7c34b3be3a6eed070a88" gracePeriod=30
Oct 30 21:09:02.946432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:02.946371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:03.032450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:03.032417    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:03.864644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:03.864609    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:04.049245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:04.049215    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:09:04.316844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:04.316806    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:04.415794 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:04.415754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:04.537410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:04.526905    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:04.882169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:04.882133    4852 patch_prober.go:29] interesting pod/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:09:04.882674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:04.882208    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:09:05.320954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:05.320921    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:06.064284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.064250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:06.244342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.244306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:06.827158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.827122    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"]
Oct 30 21:09:06.832497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.832471    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.842188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.842155    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-e07442-openshift-default-dockercfg-zxmtx\""
Oct 30 21:09:06.844587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.844556    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"]
Oct 30 21:09:06.924068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924033    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-token\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924250 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924079    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istiod-ca-cert\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924250 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924186    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-certs\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924250 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924217    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-envoy\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924248    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spmx7\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-kube-api-access-spmx7\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924345    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-data\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924402    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-socket\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924561 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924514    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-podinfo\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:06.924617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:06.924573    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-credential-socket\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.025688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.025650    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-token\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.025688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.025691    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istiod-ca-cert\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.025962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.025772    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-certs\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.025962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.025828    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-envoy\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.025962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.025880    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spmx7\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-kube-api-access-spmx7\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.025962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.025927    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-data\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.025982    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-socket\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.026044    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-podinfo\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.026092    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-credential-socket\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.026181    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-certs\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.026423    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-socket\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.026472    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-data\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.026499    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-credential-socket\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.026565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.026535    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istiod-ca-cert\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.028191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.028171    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-envoy\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.028566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.028548    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-podinfo\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.035079 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.035055    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-token\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.035846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.035827    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spmx7\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-kube-api-access-spmx7\") pod \"e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") " pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.144932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.144833    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:07.153602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.153570    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:07.315215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.315174    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:07.440031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.439953    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:07.470601 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:09:07.470551    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564948ce_6be6_44e8_97b5_04b7c1e1ac80.slice/crio-293133539ac3c8da57421ae70f7c08ad5067a81d3f6b0b5bf2adb6a555318eea WatchSource:0}: Error finding container 293133539ac3c8da57421ae70f7c08ad5067a81d3f6b0b5bf2adb6a555318eea: Status 404 returned error can't find the container with id 293133539ac3c8da57421ae70f7c08ad5067a81d3f6b0b5bf2adb6a555318eea
Oct 30 21:09:07.472380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.472357    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"]
Oct 30 21:09:07.474065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.474032    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:09:07.474184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.474099    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:09:07.474184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.474136    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:09:07.582370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.582331    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:07.882775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:07.882741    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:08.046048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.046014    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:08.122280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.122243    4852 generic.go:356] "Generic (PLEG): container finished" podID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerID="4180dab5ee7e4501af1a4ce2a349ba33aa7093387f4a7c34b3be3a6eed070a88" exitCode=0
Oct 30 21:09:08.122465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.122310    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" event={"ID":"8fdcb224-4ee2-41da-942a-6298b75d1a0b","Type":"ContainerDied","Data":"4180dab5ee7e4501af1a4ce2a349ba33aa7093387f4a7c34b3be3a6eed070a88"}
Oct 30 21:09:08.122465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.122381    4852 scope.go:122] "RemoveContainer" containerID="096e63de71a8bfc2c1acd651e381fae1784565d91b64056c23e721155d93d204"
Oct 30 21:09:08.124640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.124612    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" event={"ID":"564948ce-6be6-44e8-97b5-04b7c1e1ac80","Type":"ContainerStarted","Data":"4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181"}
Oct 30 21:09:08.124745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.124666    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" event={"ID":"564948ce-6be6-44e8-97b5-04b7c1e1ac80","Type":"ContainerStarted","Data":"293133539ac3c8da57421ae70f7c08ad5067a81d3f6b0b5bf2adb6a555318eea"}
Oct 30 21:09:08.172222 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.172192    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:08.174670 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.174637    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:09:08.174832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.174694    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:09:08.175321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.175265    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podStartSLOduration=2.175249719 podStartE2EDuration="2.175249719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:08.172705005 +0000 UTC m=+1856.287439196" watchObservedRunningTime="2026-10-30 21:09:08.175249719 +0000 UTC m=+1856.289983910"
Oct 30 21:09:08.178347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.178323    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:08.263880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.263853    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:09:08.354059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354023    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-certs\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354083    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-kube-api-access-dbggq\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-kube-api-access-dbggq\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354116    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-socket\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354178    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-data\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354258    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-credential-socket\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354286    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-envoy\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354356    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-certs" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:09:08.354454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354413    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istiod-ca-cert\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354456    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-token\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-token\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354518    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-socket" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:09:08.354672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354530    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-credential-socket" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:09:08.354672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354545    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-data" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:09:08.354871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354677    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-podinfo\") pod \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\" (UID: \"8fdcb224-4ee2-41da-942a-6298b75d1a0b\") "
Oct 30 21:09:08.354871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.354831    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istiod-ca-cert" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:09:08.355253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.355233    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.355348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.355256    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.355348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.355268    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.355348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.355280    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.355348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.355294    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.356926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.356891    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-podinfo" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:09:08.357116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.357093    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-envoy" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:09:08.357212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.357142    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-kube-api-access-dbggq" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "kube-api-access-dbggq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:09:08.357212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.357195    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-token" pod "8fdcb224-4ee2-41da-942a-6298b75d1a0b" (UID: "8fdcb224-4ee2-41da-942a-6298b75d1a0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:09:08.456406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.456298    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.456406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.456335    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbggq\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-kube-api-access-dbggq\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.456406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.456348    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.456406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.456360    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fdcb224-4ee2-41da-942a-6298b75d1a0b-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:08.480623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.480591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:08.594826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.594793    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:08.698666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:08.698624    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:09.133823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.133798    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"
Oct 30 21:09:09.134020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.133850    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb" event={"ID":"8fdcb224-4ee2-41da-942a-6298b75d1a0b","Type":"ContainerDied","Data":"c759b35237d5692ad0f6dd10df388a3afb5f0d2ad48f493e2a75d19fa982cd26"}
Oct 30 21:09:09.134020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.133908    4852 scope.go:122] "RemoveContainer" containerID="4180dab5ee7e4501af1a4ce2a349ba33aa7093387f4a7c34b3be3a6eed070a88"
Oct 30 21:09:09.178287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.178249    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:09:09.179228 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.179191    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:09:09.179716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.179692    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"]
Oct 30 21:09:09.184012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.183990    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-49c7c2-openshift-default-6c7f896fdf-gvlrb"]
Oct 30 21:09:09.439823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.439732    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:09.543413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.542452    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"]
Oct 30 21:09:09.547416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.544748    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy"
Oct 30 21:09:09.547416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.544781    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy"
Oct 30 21:09:09.547416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.544806    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy"
Oct 30 21:09:09.547416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.544814    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy"
Oct 30 21:09:09.547416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.545007    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy"
Oct 30 21:09:09.547416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.545025    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" containerName="istio-proxy"
Oct 30 21:09:09.564688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.556867    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.598406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.597588    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"]
Oct 30 21:09:09.625380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.625083    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.625380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.625156    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.625380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.625191    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.625380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.625242    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkp9p\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-kube-api-access-kkp9p\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.627155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.627135    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-e07442\"/\"agc-cred-fd5f971c3cd8\""
Oct 30 21:09:09.627318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.627297    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-e07442\"/\"routing-overlay\""
Oct 30 21:09:09.627435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.627402    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-e07442\"/\"openshift-service-ca.crt\""
Oct 30 21:09:09.627435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.627245    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-e07442\"/\"kube-root-ca.crt\""
Oct 30 21:09:09.627530 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.627146    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-e07442\"/\"payload-processing-external-model-plugins-e2e-extmod-b-e07442\""
Oct 30 21:09:09.735690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.729673    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.735690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.729739    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.735690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.729772    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.735690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.729823    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkp9p\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-kube-api-access-kkp9p\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.735690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.730865    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.735690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.731272    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.735690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.733757    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.750419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.741778    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkp9p\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-kube-api-access-kkp9p\") pod \"payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") " pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:09.943496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:09.943452    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:10.008817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.008782    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fdcb224-4ee2-41da-942a-6298b75d1a0b" path="/var/lib/kubelet/pods/8fdcb224-4ee2-41da-942a-6298b75d1a0b/volumes"
Oct 30 21:09:10.172417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.170809    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:09:10.172417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.170893    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:09:10.277339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.277312    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"]
Oct 30 21:09:10.280518 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:09:10.280487    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14561e99_cc99_4774_81b5_18d88231592b.slice/crio-0c6393f804c3f67c9f24bc5775e6fba12c92aa3c00543dba4c6e1b003513840c WatchSource:0}: Error finding container 0c6393f804c3f67c9f24bc5775e6fba12c92aa3c00543dba4c6e1b003513840c: Status 404 returned error can't find the container with id 0c6393f804c3f67c9f24bc5775e6fba12c92aa3c00543dba4c6e1b003513840c
Oct 30 21:09:10.443366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.443333    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:10.544542 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.544364    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:10.690317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.678862    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:10.703405 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.690502    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"]
Oct 30 21:09:10.713324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.712126    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:10.713631 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.711974    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"]
Oct 30 21:09:10.713828 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.713812    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"]
Oct 30 21:09:10.717767 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.717050    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-7386b4\""
Oct 30 21:09:10.717767 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.717221    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-7386b4-dockercfg-vzt8h\""
Oct 30 21:09:10.724425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.723549    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"]
Oct 30 21:09:10.724425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.723655    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:10.755690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.752915    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de41ee8b-c067-4676-a7ff-8ec975de653a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:10.755690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.753111    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:10.755690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.753231    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wz2h\" (UniqueName: \"kubernetes.io/projected/de41ee8b-c067-4676-a7ff-8ec975de653a-kube-api-access-4wz2h\") pod \"payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:10.755690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.753460    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spk9s\" (UniqueName: \"kubernetes.io/projected/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-kube-api-access-spk9s\") pod \"payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:10.869466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.860798    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spk9s\" (UniqueName: \"kubernetes.io/projected/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-kube-api-access-spk9s\") pod \"payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:10.869867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.869843    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de41ee8b-c067-4676-a7ff-8ec975de653a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:10.928417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.913564    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:10.928417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.913729    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wz2h\" (UniqueName: \"kubernetes.io/projected/de41ee8b-c067-4676-a7ff-8ec975de653a-kube-api-access-4wz2h\") pod \"payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:10.928417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.911958    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de41ee8b-c067-4676-a7ff-8ec975de653a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:10.928417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.914876    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:10.944963 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.944887    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wz2h\" (UniqueName: \"kubernetes.io/projected/de41ee8b-c067-4676-a7ff-8ec975de653a-kube-api-access-4wz2h\") pod \"payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") " pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:10.979248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:10.979173    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spk9s\" (UniqueName: \"kubernetes.io/projected/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-kube-api-access-spk9s\") pod \"payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:11.036429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.036370    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:11.187849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.187345    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:09:11.187849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.187455    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:09:11.191649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.191513    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm" event={"ID":"14561e99-cc99-4774-81b5-18d88231592b","Type":"ContainerStarted","Data":"0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6"}
Oct 30 21:09:11.191649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.191572    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm" event={"ID":"14561e99-cc99-4774-81b5-18d88231592b","Type":"ContainerStarted","Data":"0c6393f804c3f67c9f24bc5775e6fba12c92aa3c00543dba4c6e1b003513840c"}
Oct 30 21:09:11.191649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.191616    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:11.215040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.215012    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:11.215434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.215364    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm" podStartSLOduration=2.215345157 podStartE2EDuration="2.215345157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:11.207672155 +0000 UTC m=+1859.322406345" watchObservedRunningTime="2026-10-30 21:09:11.215345157 +0000 UTC m=+1859.330079348"
Oct 30 21:09:11.387477 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.385901    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"]
Oct 30 21:09:11.489197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.489169    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:11.517662 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:09:11.517631    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde41ee8b_c067_4676_a7ff_8ec975de653a.slice/crio-c458f5eea58df9ebf414bff7815c6be042605495a5268721425d55e42e7b0652 WatchSource:0}: Error finding container c458f5eea58df9ebf414bff7815c6be042605495a5268721425d55e42e7b0652: Status 404 returned error can't find the container with id c458f5eea58df9ebf414bff7815c6be042605495a5268721425d55e42e7b0652
Oct 30 21:09:11.517747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:11.517731    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"]
Oct 30 21:09:12.172892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.172854    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:09:12.173191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.173160    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:09:12.202227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.202188    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp" event={"ID":"0b0973be-c871-41d9-b1e6-9a29cc16ca3c","Type":"ContainerStarted","Data":"16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e"}
Oct 30 21:09:12.202462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.202256    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp" event={"ID":"0b0973be-c871-41d9-b1e6-9a29cc16ca3c","Type":"ContainerStarted","Data":"f0227dc4d7f4b6e54b159763a0ee00910e4ad4e25b3b54399a1d247457d9e645"}
Oct 30 21:09:12.202462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.202286    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:12.204600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.204575    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n" event={"ID":"de41ee8b-c067-4676-a7ff-8ec975de653a","Type":"ContainerStarted","Data":"b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b"}
Oct 30 21:09:12.204737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.204626    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n" event={"ID":"de41ee8b-c067-4676-a7ff-8ec975de653a","Type":"ContainerStarted","Data":"c458f5eea58df9ebf414bff7815c6be042605495a5268721425d55e42e7b0652"}
Oct 30 21:09:12.204737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.204686    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:12.229301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.229229    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp" podStartSLOduration=2.229209763 podStartE2EDuration="2.229209763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:12.22671111 +0000 UTC m=+1860.341445301" watchObservedRunningTime="2026-10-30 21:09:12.229209763 +0000 UTC m=+1860.343943955"
Oct 30 21:09:12.289422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.287983    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n" podStartSLOduration=2.287963306 podStartE2EDuration="2.287963306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:12.245841554 +0000 UTC m=+1860.360575745" watchObservedRunningTime="2026-10-30 21:09:12.287963306 +0000 UTC m=+1860.402697497"
Oct 30 21:09:12.295618 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:12.295298    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:13.158411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:13.154833    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:09:13.158411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:13.154902    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:09:13.173413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:13.173351    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:14.155200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:14.155094    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 30 21:09:14.155200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:14.155182    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 30 21:09:14.465891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:14.465772    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:15.158943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:15.158911    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:15.159320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:15.159296    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:15.160196 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:15.160095    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:09:15.325610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:15.325575    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:16.008197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:16.008164    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:17.102090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:17.102051    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:18.066914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:18.066880    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:19.244447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:19.238230    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:20.255579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:20.255548    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:21.228195 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:21.228156    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:21.330037 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:21.329992    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:21.558954 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:09:21.558909    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:09:22.171556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.171526    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:22.217696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.217653    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:09:22.451576 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.451480    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:22.793988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.793955    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"]
Oct 30 21:09:22.800131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.800110    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.811144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.811117    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"]
Oct 30 21:09:22.812365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.812339    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-d95261-openshift-default-dockercfg-qrtlx\""
Oct 30 21:09:22.838146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838111    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-credential-socket\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838186    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-socket\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838244    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-podinfo\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838360    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-certs\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838364    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:22.838497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838407    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60f52450-d4d3-4903-8be4-c97eac38e4f9-istiod-ca-cert\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838441    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-data\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838574    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqc8q\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-kube-api-access-mqc8q\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838605    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-envoy\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.838727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.838710    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-token\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.946323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.946279    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-certs\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.948264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.948237    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60f52450-d4d3-4903-8be4-c97eac38e4f9-istiod-ca-cert\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.973410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.948463    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-data\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.975895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.975863    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqc8q\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-kube-api-access-mqc8q\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.976511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.976486    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-envoy\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.977098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.951566    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-data\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.977271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.946960    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-certs\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.977439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.951276    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60f52450-d4d3-4903-8be4-c97eac38e4f9-istiod-ca-cert\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.977770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.977748    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-token\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.992784 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.992750    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-envoy\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.995779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.995747    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-credential-socket\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.998103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.978375    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-credential-socket\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.998329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.998303    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-socket\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.998666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.998633    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-podinfo\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:22.998838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:22.998811    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-socket\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:23.015415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.009993    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-podinfo\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:23.015415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.010799    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqc8q\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-kube-api-access-mqc8q\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:23.041406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.040714    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-token\") pod \"e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") " pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:23.130826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.130733    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:23.178835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.178798    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:23.226775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.226743    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:09:23.227822 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.227797    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:09:23.294924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.294885    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:23.431117 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.431088    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"]
Oct 30 21:09:23.431248 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:09:23.431134    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f52450_d4d3_4903_8be4_c97eac38e4f9.slice/crio-37638c94bbf47435e5c0aa6ec231390158578f89410bc791595d65a86c86e13b WatchSource:0}: Error finding container 37638c94bbf47435e5c0aa6ec231390158578f89410bc791595d65a86c86e13b: Status 404 returned error can't find the container with id 37638c94bbf47435e5c0aa6ec231390158578f89410bc791595d65a86c86e13b
Oct 30 21:09:23.433927 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.433898    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:09:23.434020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.433962    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:09:23.434020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.433992    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:09:23.470434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.470403    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:23.756821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.756788    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:23.857330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:23.857293    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:24.219498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:24.219371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:24.340188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:24.340141    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" event={"ID":"60f52450-d4d3-4903-8be4-c97eac38e4f9","Type":"ContainerStarted","Data":"5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb"}
Oct 30 21:09:24.340403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:24.340292    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" event={"ID":"60f52450-d4d3-4903-8be4-c97eac38e4f9","Type":"ContainerStarted","Data":"37638c94bbf47435e5c0aa6ec231390158578f89410bc791595d65a86c86e13b"}
Oct 30 21:09:24.377627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:24.377550    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podStartSLOduration=2.377530436 podStartE2EDuration="2.377530436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:24.371532976 +0000 UTC m=+1872.486267172" watchObservedRunningTime="2026-10-30 21:09:24.377530436 +0000 UTC m=+1872.492264625"
Oct 30 21:09:25.032715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:25.032645    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:25.131636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:25.131603    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:25.140786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:25.140753    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:25.348739 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:25.348657    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:25.349752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:25.349729    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:09:25.905222 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:25.905187    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:26.084170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:26.084131    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:26.182858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:26.182779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:26.552960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:26.552919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:26.644174 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:26.644133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:27.140397 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:27.140354    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:27.827850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:27.827820    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:27.932038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:27.932003    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:28.517195 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:28.517160    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:29.891360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:29.891329    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:30.724695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:30.724645    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:30.826750 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:30.826702    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:30.924834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:30.924799    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:31.832843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:31.832802    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:32.137429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:32.137334    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:32.238318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:32.238285    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:33.056623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:33.056585    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:33.843501 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:33.843458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:34.651941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:34.651906    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:35.500702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:35.500656    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:36.216934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:36.216896    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:36.798282 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:36.798246    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"]
Oct 30 21:09:36.798779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:36.798528    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px" podUID="aed980dc-f631-427f-aaa9-af833637e796" containerName="maas-api" containerID="cri-o://e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65" gracePeriod=30
Oct 30 21:09:36.838051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:36.838015    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:37.186376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.011994    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:37.262003 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.261972    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:09:37.359867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.359835    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aed980dc-f631-427f-aaa9-af833637e796-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed980dc-f631-427f-aaa9-af833637e796-openshift-service-ca\") pod \"aed980dc-f631-427f-aaa9-af833637e796\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") "
Oct 30 21:09:37.360068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.359948    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-metrics-certs\") pod \"aed980dc-f631-427f-aaa9-af833637e796\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") "
Oct 30 21:09:37.360068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.359999    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-maas-api-tls\") pod \"aed980dc-f631-427f-aaa9-af833637e796\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") "
Oct 30 21:09:37.360199 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.360181    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed980dc-f631-427f-aaa9-af833637e796-kube-api-access-7qdgz\" (UniqueName: \"kubernetes.io/projected/aed980dc-f631-427f-aaa9-af833637e796-kube-api-access-7qdgz\") pod \"aed980dc-f631-427f-aaa9-af833637e796\" (UID: \"aed980dc-f631-427f-aaa9-af833637e796\") "
Oct 30 21:09:37.360274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.360246    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed980dc-f631-427f-aaa9-af833637e796-openshift-service-ca" pod "aed980dc-f631-427f-aaa9-af833637e796" (UID: "aed980dc-f631-427f-aaa9-af833637e796"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:09:37.360960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.360932    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aed980dc-f631-427f-aaa9-af833637e796-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:37.363432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.362327    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-metrics-certs" pod "aed980dc-f631-427f-aaa9-af833637e796" (UID: "aed980dc-f631-427f-aaa9-af833637e796"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:09:37.363432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.362332    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed980dc-f631-427f-aaa9-af833637e796-kube-api-access-7qdgz" pod "aed980dc-f631-427f-aaa9-af833637e796" (UID: "aed980dc-f631-427f-aaa9-af833637e796"). InnerVolumeSpecName "kube-api-access-7qdgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:09:37.363432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.362584    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-maas-api-tls" pod "aed980dc-f631-427f-aaa9-af833637e796" (UID: "aed980dc-f631-427f-aaa9-af833637e796"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:09:37.370325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.370291    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:37.480032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.471676    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:37.480032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.479619    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aed980dc-f631-427f-aaa9-af833637e796-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:37.480032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.479639    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qdgz\" (UniqueName: \"kubernetes.io/projected/aed980dc-f631-427f-aaa9-af833637e796-kube-api-access-7qdgz\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:37.480032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.479521    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:37.530419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.526911    4852 generic.go:356] "Generic (PLEG): container finished" podID="aed980dc-f631-427f-aaa9-af833637e796" containerID="e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65" exitCode=0
Oct 30 21:09:37.530419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.527220    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px" event={"ID":"aed980dc-f631-427f-aaa9-af833637e796","Type":"ContainerDied","Data":"e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65"}
Oct 30 21:09:37.530419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.527296    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px" event={"ID":"aed980dc-f631-427f-aaa9-af833637e796","Type":"ContainerDied","Data":"21d2567087fc4c08281fd1fc12739decfdc525fc3224b36b53f175354dd0f716"}
Oct 30 21:09:37.530419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.527324    4852 scope.go:122] "RemoveContainer" containerID="e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65"
Oct 30 21:09:37.530419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.527512    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"
Oct 30 21:09:37.545913 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.545888    4852 scope.go:122] "RemoveContainer" containerID="e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65"
Oct 30 21:09:37.546226 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:09:37.546205    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65\": container with ID starting with e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65 not found: ID does not exist" containerID="e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65"
Oct 30 21:09:37.546292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.546235    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65"} err="failed to get container status \"e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65\": rpc error: code = NotFound desc = could not find container \"e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65\": container with ID starting with e3e407b4f372f243ff0d90f988115aea4ec9ce96c7fb3539fd0840ab48a09e65 not found: ID does not exist"
Oct 30 21:09:37.560703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.560672    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"]
Oct 30 21:09:37.565720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.565694    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-435414-895f5d99c-ng5px"]
Oct 30 21:09:37.737945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:37.737846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:38.006575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:38.006550    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aed980dc-f631-427f-aaa9-af833637e796" path="/var/lib/kubelet/pods/aed980dc-f631-427f-aaa9-af833637e796/volumes"
Oct 30 21:09:38.542171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:38.542137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:38.773833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:38.773795    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:38.884065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:38.883982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:39.023095 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:39.023060    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:39.185632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:39.185544    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:39.732016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:39.731980    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:39.829325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:39.829292    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:40.605359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:40.605319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:40.793408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:40.778965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:41.145346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:41.145309    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:42.100554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:42.100499    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:43.122119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:43.122082    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.054400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:44.054365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.512707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:44.512675    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.612807 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:44.612763    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:44.851284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:44.851184    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:45.050688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:45.050632    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:46.042543 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.042510    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:46.773178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.773133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:46.904263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.904224    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"]
Oct 30 21:09:46.905346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.905315    4852 cpu_manager.go:414] "containerMap: removing container" podUID="aed980dc-f631-427f-aaa9-af833637e796" containerName="maas-api"
Oct 30 21:09:46.905346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.905343    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aed980dc-f631-427f-aaa9-af833637e796" containerName="maas-api"
Oct 30 21:09:46.905623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.905604    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="aed980dc-f631-427f-aaa9-af833637e796" containerName="maas-api"
Oct 30 21:09:46.912086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.912059    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:46.919688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.919661    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-d95261-serving-cert\""
Oct 30 21:09:46.920986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.920960    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"]
Oct 30 21:09:46.954946 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.954911    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-metrics-certs\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:46.955110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.954991    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:46.955190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.955165    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-maas-api-tls\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:46.955247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.955202    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftvdh\" (UniqueName: \"kubernetes.io/projected/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-kube-api-access-ftvdh\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:46.958794 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:46.958754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:47.069405 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.068735    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-metrics-certs\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.069405 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.068821    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.069405 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.068984    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-maas-api-tls\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.069405 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.069011    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftvdh\" (UniqueName: \"kubernetes.io/projected/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-kube-api-access-ftvdh\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.071827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.071775    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.075098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.075065    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:47.079489 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.079459    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-maas-api-tls\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.084144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.084110    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-metrics-certs\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.091314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.091246    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftvdh\" (UniqueName: \"kubernetes.io/projected/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-kube-api-access-ftvdh\") pod \"maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.239559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.239519    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:47.535167 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.535111    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:47.581106 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:09:47.581058    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cd2ca5_85eb_456f_b1a9_c3b8fc44fb47.slice/crio-9be873583e9a61ff9c3932a14e0a32e2ea14e109f25387bacb256feeeb0a6b31 WatchSource:0}: Error finding container 9be873583e9a61ff9c3932a14e0a32e2ea14e109f25387bacb256feeeb0a6b31: Status 404 returned error can't find the container with id 9be873583e9a61ff9c3932a14e0a32e2ea14e109f25387bacb256feeeb0a6b31
Oct 30 21:09:47.582052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.582021    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"]
Oct 30 21:09:47.618042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.618006    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b" event={"ID":"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47","Type":"ContainerStarted","Data":"9be873583e9a61ff9c3932a14e0a32e2ea14e109f25387bacb256feeeb0a6b31"}
Oct 30 21:09:47.624539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:47.624469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:48.230900 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:48.230866    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:48.628468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:48.628431    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b" event={"ID":"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47","Type":"ContainerStarted","Data":"ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e"}
Oct 30 21:09:48.628656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:48.628515    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:48.651022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:48.650971    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b" podStartSLOduration=2.216183723 podStartE2EDuration="2.65095725s" totalImagesPullingTime="434.773527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:09:48.647226147 +0000 UTC m=+1896.761960362" watchObservedRunningTime="2026-10-30 21:09:48.65095725 +0000 UTC m=+1896.765691439"
Oct 30 21:09:49.308050 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:49.308003    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:50.045941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:50.045905    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:51.140306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:51.140272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:51.236269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:51.236236    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:51.562177 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:09:51.562141    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:09:52.146107 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:52.146071    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.507974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:52.507933    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.603569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:52.603520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:53.492115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:53.492077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:54.277226 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:54.277190    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:54.389795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:54.389757    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:54.642770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:54.642697    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:09:55.324086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:55.324052    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:56.021677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:56.021642    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:56.740154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:56.740092    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:57.781607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:57.781566    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:57.876606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:57.876574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:58.861259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:58.861226    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:59.996436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:09:59.996373    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:00.905270 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:00.905238    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:01.047810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:01.047775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:02.140091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:02.140055    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:02.962538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:02.962504    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:03.867804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:03.867759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:04.462951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:04.462915    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:04.712285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:04.712249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:05.413454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:05.413417    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:05.572145 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:05.572114    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:06.724445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:06.724382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:07.538453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:07.538412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:08.577970 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:08.577923    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:09.526127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:09.526076    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:10.334967 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:10.334924    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:11.322878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:11.322836    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:11.424865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:11.424824    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:11.521908 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:11.521873    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:12.220905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:12.220865    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:12.451038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:12.451000    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:13.324342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:13.324262    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:13.424900 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:13.424860    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:13.539949 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:13.539917    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:13.630268 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:13.630153    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:13.839943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:13.839895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:13.985107 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:13.985073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:14.081880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:14.081828    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:14.520494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:14.520464    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:14.621553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:14.621516    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:15.330271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:15.330238    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:15.434758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:15.434723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:15.971808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:15.971772    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"]
Oct 30 21:10:15.986321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:15.986290    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"]
Oct 30 21:10:16.039928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:16.039892    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:16.124246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:16.124209    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:16.626764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:16.626729    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:17.465002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:17.464956    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:18.221824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:18.221792    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:18.916359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:18.916324    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:19.635288 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:19.635254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:20.308826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:20.308787    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:20.972465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:20.972420    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n" podUID="de41ee8b-c067-4676-a7ff-8ec975de653a" containerName="payload-processing" containerID="cri-o://b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b" gracePeriod=25
Oct 30 21:10:20.987074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:20.987015    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp" podUID="0b0973be-c871-41d9-b1e6-9a29cc16ca3c" containerName="payload-pre-processing" containerID="cri-o://16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e" gracePeriod=25
Oct 30 21:10:21.015636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.015605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:21.338348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.338322    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:10:21.341991 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.341968    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:10:21.405359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.405314    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de41ee8b-c067-4676-a7ff-8ec975de653a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de41ee8b-c067-4676-a7ff-8ec975de653a-plugins-config-volume\") pod \"de41ee8b-c067-4676-a7ff-8ec975de653a\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") "
Oct 30 21:10:21.405551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.405534    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de41ee8b-c067-4676-a7ff-8ec975de653a-kube-api-access-4wz2h\" (UniqueName: \"kubernetes.io/projected/de41ee8b-c067-4676-a7ff-8ec975de653a-kube-api-access-4wz2h\") pod \"de41ee8b-c067-4676-a7ff-8ec975de653a\" (UID: \"de41ee8b-c067-4676-a7ff-8ec975de653a\") "
Oct 30 21:10:21.405614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.405587    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-plugins-config-volume\") pod \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") "
Oct 30 21:10:21.405683 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.405632    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-kube-api-access-spk9s\" (UniqueName: \"kubernetes.io/projected/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-kube-api-access-spk9s\") pod \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\" (UID: \"0b0973be-c871-41d9-b1e6-9a29cc16ca3c\") "
Oct 30 21:10:21.405865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.405833    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de41ee8b-c067-4676-a7ff-8ec975de653a-plugins-config-volume" pod "de41ee8b-c067-4676-a7ff-8ec975de653a" (UID: "de41ee8b-c067-4676-a7ff-8ec975de653a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:10:21.406102 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.406076    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-plugins-config-volume" pod "0b0973be-c871-41d9-b1e6-9a29cc16ca3c" (UID: "0b0973be-c871-41d9-b1e6-9a29cc16ca3c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:10:21.406598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.406573    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:21.406734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.406602    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de41ee8b-c067-4676-a7ff-8ec975de653a-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:21.408154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.408129    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de41ee8b-c067-4676-a7ff-8ec975de653a-kube-api-access-4wz2h" pod "de41ee8b-c067-4676-a7ff-8ec975de653a" (UID: "de41ee8b-c067-4676-a7ff-8ec975de653a"). InnerVolumeSpecName "kube-api-access-4wz2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:10:21.408436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.408416    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-kube-api-access-spk9s" pod "0b0973be-c871-41d9-b1e6-9a29cc16ca3c" (UID: "0b0973be-c871-41d9-b1e6-9a29cc16ca3c"). InnerVolumeSpecName "kube-api-access-spk9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:10:21.508187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.508148    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wz2h\" (UniqueName: \"kubernetes.io/projected/de41ee8b-c067-4676-a7ff-8ec975de653a-kube-api-access-4wz2h\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:21.508187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.508181    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spk9s\" (UniqueName: \"kubernetes.io/projected/0b0973be-c871-41d9-b1e6-9a29cc16ca3c-kube-api-access-spk9s\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:21.560889 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:10:21.560733    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:10:21.753288 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.753250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:21.936762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.936660    4852 generic.go:356] "Generic (PLEG): container finished" podID="0b0973be-c871-41d9-b1e6-9a29cc16ca3c" containerID="16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e" exitCode=0
Oct 30 21:10:21.936762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.936711    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"
Oct 30 21:10:21.936762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.936745    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp" event={"ID":"0b0973be-c871-41d9-b1e6-9a29cc16ca3c","Type":"ContainerDied","Data":"16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e"}
Oct 30 21:10:21.937046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.936814    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp" event={"ID":"0b0973be-c871-41d9-b1e6-9a29cc16ca3c","Type":"ContainerDied","Data":"f0227dc4d7f4b6e54b159763a0ee00910e4ad4e25b3b54399a1d247457d9e645"}
Oct 30 21:10:21.937046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.936849    4852 scope.go:122] "RemoveContainer" containerID="16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e"
Oct 30 21:10:21.938283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.938259    4852 generic.go:356] "Generic (PLEG): container finished" podID="de41ee8b-c067-4676-a7ff-8ec975de653a" containerID="b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b" exitCode=0
Oct 30 21:10:21.938482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.938319    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"
Oct 30 21:10:21.938482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.938342    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n" event={"ID":"de41ee8b-c067-4676-a7ff-8ec975de653a","Type":"ContainerDied","Data":"b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b"}
Oct 30 21:10:21.938622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.938491    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n" event={"ID":"de41ee8b-c067-4676-a7ff-8ec975de653a","Type":"ContainerDied","Data":"c458f5eea58df9ebf414bff7815c6be042605495a5268721425d55e42e7b0652"}
Oct 30 21:10:21.950399 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.950370    4852 scope.go:122] "RemoveContainer" containerID="16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e"
Oct 30 21:10:21.950776 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:10:21.950752    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e\": container with ID starting with 16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e not found: ID does not exist" containerID="16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e"
Oct 30 21:10:21.950877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.950787    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e"} err="failed to get container status \"16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e\": rpc error: code = NotFound desc = could not find container \"16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e\": container with ID starting with 16e899ea6920786d80710c7257151ba11d96b3b643d742bcf13ac19478b8a63e not found: ID does not exist"
Oct 30 21:10:21.950877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.950810    4852 scope.go:122] "RemoveContainer" containerID="b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b"
Oct 30 21:10:21.961938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.961918    4852 scope.go:122] "RemoveContainer" containerID="b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b"
Oct 30 21:10:21.962229 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:10:21.962205    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b\": container with ID starting with b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b not found: ID does not exist" containerID="b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b"
Oct 30 21:10:21.962285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.962234    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b"} err="failed to get container status \"b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b\": rpc error: code = NotFound desc = could not find container \"b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b\": container with ID starting with b2b28a536f2121a71da5bb0b642b4301bb792c7d3fdff461f48198c60d69759b not found: ID does not exist"
Oct 30 21:10:21.975931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.975898    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"]
Oct 30 21:10:21.980559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.980530    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-7386b4-5d77f9646f-jgf2n"]
Oct 30 21:10:21.996231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:21.996200    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"]
Oct 30 21:10:22.006591 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:22.006556    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de41ee8b-c067-4676-a7ff-8ec975de653a" path="/var/lib/kubelet/pods/de41ee8b-c067-4676-a7ff-8ec975de653a/volumes"
Oct 30 21:10:22.006997 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:22.006975    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-7386b4-69d55df66c-ftfdp"]
Oct 30 21:10:22.639018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:22.638970    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:23.373069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:23.373031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:24.005729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:24.005698    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b0973be-c871-41d9-b1e6-9a29cc16ca3c" path="/var/lib/kubelet/pods/0b0973be-c871-41d9-b1e6-9a29cc16ca3c/volumes"
Oct 30 21:10:24.155759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:24.155722    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:25.123175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:25.123136    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:25.932159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:25.932122    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:26.649733 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:26.649689    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:27.856609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:27.856571    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:29.009335 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:29.009304    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:29.942116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:29.942082    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:30.831137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:30.831106    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:31.723448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:31.723356    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:32.774263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:32.774229    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:33.556665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:33.556629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:33.783705 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:33.783666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:33.879046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:33.878927    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:34.147667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:34.147541    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:34.524735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:34.524681    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:35.258885 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:35.258826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:36.067940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:36.067904    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:36.876665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:36.876629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:36.977843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:36.977803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:37.637829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:37.637796    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:38.274047 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.273991    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:38.430789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.430754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:38.679629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.679538    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"]
Oct 30 21:10:38.680960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.680912    4852 cpu_manager.go:414] "containerMap: removing container" podUID="de41ee8b-c067-4676-a7ff-8ec975de653a" containerName="payload-processing"
Oct 30 21:10:38.680960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.680949    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de41ee8b-c067-4676-a7ff-8ec975de653a" containerName="payload-processing"
Oct 30 21:10:38.681173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.681039    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0b0973be-c871-41d9-b1e6-9a29cc16ca3c" containerName="payload-pre-processing"
Oct 30 21:10:38.681173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.681050    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b0973be-c871-41d9-b1e6-9a29cc16ca3c" containerName="payload-pre-processing"
Oct 30 21:10:38.681292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.681255    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="de41ee8b-c067-4676-a7ff-8ec975de653a" containerName="payload-processing"
Oct 30 21:10:38.681292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.681284    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b0973be-c871-41d9-b1e6-9a29cc16ca3c" containerName="payload-pre-processing"
Oct 30 21:10:38.689524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.689491    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.693849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.693818    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"]
Oct 30 21:10:38.697253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.697231    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-e1e0f5-openshift-default-dockercfg-jmrsc\""
Oct 30 21:10:38.719244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.719213    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:38.853408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.853242    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854288    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-socket\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854337    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-token\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854406    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lhmf\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-kube-api-access-2lhmf\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854447    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-envoy\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854524    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-certs\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854548    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-data\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854784    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-credential-socket\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854842    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istiod-ca-cert\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.857404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.854872    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-podinfo\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.966466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.966347    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istiod-ca-cert\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.984722    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-podinfo\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.984834    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-socket\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.984864    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-token\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.984914    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lhmf\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-kube-api-access-2lhmf\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.984964    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-envoy\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.985085    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-certs\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.985121    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-data\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.985468    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-credential-socket\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.986043    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-credential-socket\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.988461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.967563    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istiod-ca-cert\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.989792 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.989762    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-certs\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.990043 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.990022    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-socket\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.990920 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.990770    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-data\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:38.995291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:38.995267    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-envoy\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:39.003946 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.003044    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-token\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:39.005235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.005183    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lhmf\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-kube-api-access-2lhmf\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:39.006002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.005944    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-podinfo\") pod \"e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") " pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:39.008884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.008664    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:39.318205 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:10:39.318163    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f57cfad_f8d4_4b39_a024_dfefeba9b431.slice/crio-35b825fe5988d8a5e0a45fabeb1d615079fdc98faad879b3ad0c1cb2c1f1e03a WatchSource:0}: Error finding container 35b825fe5988d8a5e0a45fabeb1d615079fdc98faad879b3ad0c1cb2c1f1e03a: Status 404 returned error can't find the container with id 35b825fe5988d8a5e0a45fabeb1d615079fdc98faad879b3ad0c1cb2c1f1e03a
Oct 30 21:10:39.319487 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.319460    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"]
Oct 30 21:10:39.321425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.321374    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:10:39.321513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.321462    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:10:39.321513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.321493    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:10:39.613133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:39.613043    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:40.164785 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:40.164738    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" event={"ID":"5f57cfad-f8d4-4b39-a024-dfefeba9b431","Type":"ContainerStarted","Data":"1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f"}
Oct 30 21:10:40.164995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:40.164806    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" event={"ID":"5f57cfad-f8d4-4b39-a024-dfefeba9b431","Type":"ContainerStarted","Data":"35b825fe5988d8a5e0a45fabeb1d615079fdc98faad879b3ad0c1cb2c1f1e03a"}
Oct 30 21:10:40.196121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:40.196061    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podStartSLOduration=2.196045752 podStartE2EDuration="2.196045752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:10:40.191552144 +0000 UTC m=+1948.306286334" watchObservedRunningTime="2026-10-30 21:10:40.196045752 +0000 UTC m=+1948.310780001"
Oct 30 21:10:40.502921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:40.502827    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:40.804759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:40.804721    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"]
Oct 30 21:10:40.840542 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:40.840505    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:40.956301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:40.956266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:41.010015 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:41.009855    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:41.031293 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:41.030412    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:41.175881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:41.175784    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:41.177048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:41.177018    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:10:41.403061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:41.403031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:41.842167 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:41.842128    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:42.015961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:42.015931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:42.186754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:42.186646    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:42.279670 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:42.279636    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:42.532464 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:42.532428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:42.693276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:42.693242    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:43.163445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:43.163408    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:43.778697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:43.778659    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:43.875803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:43.875765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:44.083465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:44.077919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:44.445846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:44.445730    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:44.579410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:44.575521    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:44.780221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:44.780058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:45.590804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:45.590711    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:45.805497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:45.805437    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm" podUID="14561e99-cc99-4774-81b5-18d88231592b" containerName="payload-processing" containerID="cri-o://0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6" gracePeriod=25
Oct 30 21:10:46.132325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.132300    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:10:46.167397 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.167344    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-plugins-config-volume\") pod \"14561e99-cc99-4774-81b5-18d88231592b\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") "
Oct 30 21:10:46.167594 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.167486    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-kube-api-access-kkp9p\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-kube-api-access-kkp9p\") pod \"14561e99-cc99-4774-81b5-18d88231592b\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") "
Oct 30 21:10:46.167679 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.167591    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-routing-overlay-volume\") pod \"14561e99-cc99-4774-81b5-18d88231592b\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") "
Oct 30 21:10:46.167679 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.167631    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-provider-credentials\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-provider-credentials\") pod \"14561e99-cc99-4774-81b5-18d88231592b\" (UID: \"14561e99-cc99-4774-81b5-18d88231592b\") "
Oct 30 21:10:46.167849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.167801    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-plugins-config-volume" pod "14561e99-cc99-4774-81b5-18d88231592b" (UID: "14561e99-cc99-4774-81b5-18d88231592b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:10:46.168325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.168300    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:46.168495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.168302    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-routing-overlay-volume" pod "14561e99-cc99-4774-81b5-18d88231592b" (UID: "14561e99-cc99-4774-81b5-18d88231592b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:10:46.170712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.170683    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-kube-api-access-kkp9p" pod "14561e99-cc99-4774-81b5-18d88231592b" (UID: "14561e99-cc99-4774-81b5-18d88231592b"). InnerVolumeSpecName "kube-api-access-kkp9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:10:46.170906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.170885    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-provider-credentials" pod "14561e99-cc99-4774-81b5-18d88231592b" (UID: "14561e99-cc99-4774-81b5-18d88231592b"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:10:46.216517 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.216484    4852 generic.go:356] "Generic (PLEG): container finished" podID="14561e99-cc99-4774-81b5-18d88231592b" containerID="0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6" exitCode=0
Oct 30 21:10:46.216760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.216539    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"
Oct 30 21:10:46.216760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.216562    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm" event={"ID":"14561e99-cc99-4774-81b5-18d88231592b","Type":"ContainerDied","Data":"0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6"}
Oct 30 21:10:46.216760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.216616    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm" event={"ID":"14561e99-cc99-4774-81b5-18d88231592b","Type":"ContainerDied","Data":"0c6393f804c3f67c9f24bc5775e6fba12c92aa3c00543dba4c6e1b003513840c"}
Oct 30 21:10:46.216760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.216639    4852 scope.go:122] "RemoveContainer" containerID="0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6"
Oct 30 21:10:46.231861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.231839    4852 scope.go:122] "RemoveContainer" containerID="0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6"
Oct 30 21:10:46.232170 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:10:46.232145    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6\": container with ID starting with 0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6 not found: ID does not exist" containerID="0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6"
Oct 30 21:10:46.232305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.232176    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6"} err="failed to get container status \"0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6\": rpc error: code = NotFound desc = could not find container \"0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6\": container with ID starting with 0640f727d116a999e9b760d429ce41d593ba3c255b7690f86497cfbbd2a913d6 not found: ID does not exist"
Oct 30 21:10:46.247091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.247057    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"]
Oct 30 21:10:46.251291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.251261    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-e07442/payload-processing-external-model-e2e-extmod-b-e07442-587ccrcmm"]
Oct 30 21:10:46.270074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.270042    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkp9p\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-kube-api-access-kkp9p\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:46.270074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.270072    4852 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/14561e99-cc99-4774-81b5-18d88231592b-routing-overlay-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:46.270266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.270086    4852 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/14561e99-cc99-4774-81b5-18d88231592b-provider-credentials\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:10:46.459612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:46.459528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:47.611612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:47.607104    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:47.969872 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:47.969793    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:48.007708 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:48.007674    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14561e99-cc99-4774-81b5-18d88231592b" path="/var/lib/kubelet/pods/14561e99-cc99-4774-81b5-18d88231592b/volumes"
Oct 30 21:10:48.071174 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:48.071144    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:48.433646 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:48.433610    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:48.533889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:48.533857    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:48.634698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:48.634666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:49.208579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.208540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:49.321367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.321331    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:49.327377 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.327347    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"]
Oct 30 21:10:49.328490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.328470    4852 cpu_manager.go:414] "containerMap: removing container" podUID="14561e99-cc99-4774-81b5-18d88231592b" containerName="payload-processing"
Oct 30 21:10:49.328552 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.328493    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14561e99-cc99-4774-81b5-18d88231592b" containerName="payload-processing"
Oct 30 21:10:49.328688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.328674    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="14561e99-cc99-4774-81b5-18d88231592b" containerName="payload-processing"
Oct 30 21:10:49.334365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.334341    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.343437 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.343415    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-00909f-dockercfg-qxxjm\""
Oct 30 21:10:49.345870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.345848    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"]
Oct 30 21:10:49.424730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.417101    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-00909f\""
Oct 30 21:10:49.471463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.468460    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7m7h\" (UniqueName: \"kubernetes.io/projected/4e636851-f7e8-46d2-bd43-f7202a0fcb32-kube-api-access-p7m7h\") pod \"payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.471463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.468526    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e636851-f7e8-46d2-bd43-f7202a0fcb32-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.487759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.487724    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:49.570156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.570121    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7m7h\" (UniqueName: \"kubernetes.io/projected/4e636851-f7e8-46d2-bd43-f7202a0fcb32-kube-api-access-p7m7h\") pod \"payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.570976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.570948    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e636851-f7e8-46d2-bd43-f7202a0fcb32-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.572511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.572099    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e636851-f7e8-46d2-bd43-f7202a0fcb32-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.612064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.612016    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7m7h\" (UniqueName: \"kubernetes.io/projected/4e636851-f7e8-46d2-bd43-f7202a0fcb32-kube-api-access-p7m7h\") pod \"payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.653332 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.653296    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:49.954710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:49.954689    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"]
Oct 30 21:10:50.216448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:50.216378    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:50.253105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:50.253014    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" event={"ID":"4e636851-f7e8-46d2-bd43-f7202a0fcb32","Type":"ContainerStarted","Data":"45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c"}
Oct 30 21:10:50.253105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:50.253081    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:10:50.253366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:50.253106    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" event={"ID":"4e636851-f7e8-46d2-bd43-f7202a0fcb32","Type":"ContainerStarted","Data":"a7f8ece0b759bd713312f6e0ffb2a22b3fc107c34f5ff0b612df1334691173fe"}
Oct 30 21:10:50.298993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:50.298929    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" podStartSLOduration=1.29890378 podStartE2EDuration="1.29890378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:10:50.272560323 +0000 UTC m=+1958.387294515" watchObservedRunningTime="2026-10-30 21:10:50.29890378 +0000 UTC m=+1958.413637963"
Oct 30 21:10:50.927796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:50.927756    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:51.420337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:51.420305    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:51.559927 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:10:51.559891    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:10:51.791146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:51.791114    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:52.645081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:52.645046    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:53.596422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:53.593926    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:54.213876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:54.213841    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:54.999736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:54.999696    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:55.512700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:55.512649    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:55.613344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:55.613303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:55.746436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:55.746374    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:55.952796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:55.952708    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:56.098101 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:56.098067    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:57.049445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:57.049408    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:57.195511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:57.195476    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:58.021489 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:58.021456    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:58.782302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:58.782267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:59.547858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:10:59.547819    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:00.451153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:00.451091    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:01.265239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:01.265212    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:11:01.589278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:01.589199    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:01.691065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:01.691031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:01.819860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:01.819813    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:02.732977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:02.732939    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:02.984816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:02.984736    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:03.608158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:03.608111    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:04.322310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:04.322276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:04.511603 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:04.511570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.110714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:05.110680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.810574 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:05.810542    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:06.603143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.603108    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"]
Oct 30 21:11:06.608869 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.608843    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.630762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.630731    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-00909f-serving-cert\""
Oct 30 21:11:06.633364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.633327    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrbvf\" (UniqueName: \"kubernetes.io/projected/3a0d7c61-76d4-4830-9181-5875507551f3-kube-api-access-mrbvf\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.633600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.633440    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.633684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.633651    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-metrics-certs\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.633748 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.633713    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d7c61-76d4-4830-9181-5875507551f3-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.638320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.638291    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"]
Oct 30 21:11:06.653747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.653716    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:06.749656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.739121    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-metrics-certs\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.750717 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.750684    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d7c61-76d4-4830-9181-5875507551f3-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.754858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.750910    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbvf\" (UniqueName: \"kubernetes.io/projected/3a0d7c61-76d4-4830-9181-5875507551f3-kube-api-access-mrbvf\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.755760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.755667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:06.756103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.756078    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.757055 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:11:06.756305    4852 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-serving-cert: secret "maas-api-e2e-extmod-a-00909f-serving-cert" not found
Oct 30 21:11:06.757341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.756801    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-metrics-certs\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.757571 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:11:06.757526    4852 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls podName:3a0d7c61-76d4-4830-9181-5875507551f3 nodeName:}" failed. No retries permitted until 2026-10-30 21:11:07.257223529 +0000 UTC m=+1975.371957697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls") pod "maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv" (UID: "3a0d7c61-76d4-4830-9181-5875507551f3") : secret "maas-api-e2e-extmod-a-00909f-serving-cert" not found
Oct 30 21:11:06.777795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.770868    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d7c61-76d4-4830-9181-5875507551f3-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.789295 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.785483    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrbvf\" (UniqueName: \"kubernetes.io/projected/3a0d7c61-76d4-4830-9181-5875507551f3-kube-api-access-mrbvf\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:06.879470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:06.879342    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:07.263485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:07.263451    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:07.265989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:07.265959    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls\") pod \"maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:07.548534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:07.548452    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:07.729069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:07.729034    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:07.842426 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:11:07.842378    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0d7c61_76d4_4830_9181_5875507551f3.slice/crio-3bc3d72c0e65d8f76b21ba41202dc0a7f81cb1cfdb2c1abc7cd29460dd8592fc WatchSource:0}: Error finding container 3bc3d72c0e65d8f76b21ba41202dc0a7f81cb1cfdb2c1abc7cd29460dd8592fc: Status 404 returned error can't find the container with id 3bc3d72c0e65d8f76b21ba41202dc0a7f81cb1cfdb2c1abc7cd29460dd8592fc
Oct 30 21:11:07.842743 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:07.842718    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"]
Oct 30 21:11:08.506544 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:08.506502    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv" event={"ID":"3a0d7c61-76d4-4830-9181-5875507551f3","Type":"ContainerStarted","Data":"7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204"}
Oct 30 21:11:08.507009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:08.506569    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv" event={"ID":"3a0d7c61-76d4-4830-9181-5875507551f3","Type":"ContainerStarted","Data":"3bc3d72c0e65d8f76b21ba41202dc0a7f81cb1cfdb2c1abc7cd29460dd8592fc"}
Oct 30 21:11:08.507009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:08.506777    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:08.508342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:08.508302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:08.533644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:08.533590    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv" podStartSLOduration=2.158558308 podStartE2EDuration="2.533574601s" totalImagesPullingTime="375.016293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:08.527850732 +0000 UTC m=+1976.642584923" watchObservedRunningTime="2026-10-30 21:11:08.533574601 +0000 UTC m=+1976.648308792"
Oct 30 21:11:08.613538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:08.613500    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:09.712374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:09.712337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:10.092686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:10.092655    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:10.514363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:10.514330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:11.250167 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:11.250114    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:12.018750 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:12.018714    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:12.719019 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:12.718980    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:13.434673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:13.434631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.224730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:14.224634    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.521235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:14.521202    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:11:14.914173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:14.914079    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:15.642312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:15.642268    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.172599 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:16.172557    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.264748 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:16.264715    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.432827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:16.432729    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.559563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:16.559523    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:17.189670 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:17.189636    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:17.946166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:17.946134    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:18.804515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:18.804476    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:19.582671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:19.582622    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:20.443236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:20.443195    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:21.495774 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:21.495735    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:21.559448 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:11:21.559412    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:11:21.602063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:21.601984    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:23.111270 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:23.111239    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:23.231637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:23.231602    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:23.331166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:23.331117    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:24.146851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:24.146812    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:24.272699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:24.272661    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:24.373737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:24.373697    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:24.618120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:24.618086    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:24.860427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:24.860354    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:25.008695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:25.008666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:25.112479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:25.112445    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:25.209372 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:25.209337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:25.584896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:25.584845    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:25.686497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:25.686466    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:26.283791 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:26.283737    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:26.401256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:26.401227    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:26.972775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:26.972738    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"]
Oct 30 21:11:26.977626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:26.977604    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:26.982083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:26.982058    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-a574c2-openshift-default-dockercfg-pm5sv\""
Oct 30 21:11:26.987147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:26.987115    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"]
Oct 30 21:11:27.037355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037317    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5664b217-4b0f-434a-8606-123811d27c4b-istiod-ca-cert\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037356    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-envoy\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037659 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037401    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5664b217-4b0f-434a-8606-123811d27c4b-istio-podinfo\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037659 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037534    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-socket\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037659 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037578    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-istio-token\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037683    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-credential-socket\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037809    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-certs\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037832    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-data\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.037877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.037860    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wqt8\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-kube-api-access-9wqt8\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.139583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139548    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-certs\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.139583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139584    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-data\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.139802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139608    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wqt8\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-kube-api-access-9wqt8\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.139802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139783    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5664b217-4b0f-434a-8606-123811d27c4b-istiod-ca-cert\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.139913 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139803    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-envoy\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.139913 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139831    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5664b217-4b0f-434a-8606-123811d27c4b-istio-podinfo\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.139913 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139881    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-socket\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.140069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139915    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-istio-token\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.140069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139972    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-credential-socket\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.140069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.139973    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-certs\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.140237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.140055    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-data\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.140348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.140324    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-credential-socket\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.140532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.140512    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-socket\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.140688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.140667    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5664b217-4b0f-434a-8606-123811d27c4b-istiod-ca-cert\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.142582 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.142558    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-envoy\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.142832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.142808    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5664b217-4b0f-434a-8606-123811d27c4b-istio-podinfo\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.147780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.147758    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-istio-token\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.147961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.147943    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wqt8\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-kube-api-access-9wqt8\") pod \"e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") " pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.211864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.211831    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:27.296867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.296562    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:27.608508 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:11:27.608470    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5664b217_4b0f_434a_8606_123811d27c4b.slice/crio-873d232cf86a0c5c19e8183e7cb5a51727c84048a572addad470859147261ec7 WatchSource:0}: Error finding container 873d232cf86a0c5c19e8183e7cb5a51727c84048a572addad470859147261ec7: Status 404 returned error can't find the container with id 873d232cf86a0c5c19e8183e7cb5a51727c84048a572addad470859147261ec7
Oct 30 21:11:27.608764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.608742    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"]
Oct 30 21:11:27.612099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.612064    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:27.612207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.612140    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:27.612207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.612169    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:27.726823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.726785    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" event={"ID":"5664b217-4b0f-434a-8606-123811d27c4b","Type":"ContainerStarted","Data":"13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5"}
Oct 30 21:11:27.726999 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.726852    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" event={"ID":"5664b217-4b0f-434a-8606-123811d27c4b","Type":"ContainerStarted","Data":"873d232cf86a0c5c19e8183e7cb5a51727c84048a572addad470859147261ec7"}
Oct 30 21:11:27.746512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:27.746441    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podStartSLOduration=1.746422009 podStartE2EDuration="1.746422009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:27.744930332 +0000 UTC m=+1995.859664546" watchObservedRunningTime="2026-10-30 21:11:27.746422009 +0000 UTC m=+1995.861156202"
Oct 30 21:11:28.044407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:28.041482    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:28.315691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:28.312447    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:28.315691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:28.312612    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:11:28.315691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:28.312664    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 21:11:28.617737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:28.616484    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:29.317407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:29.314990    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:11:29.317407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:29.315060    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 21:11:29.926504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:29.926427    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.024404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.023566    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.218713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.214899    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.301413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.297832    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:11:30.301413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.297897    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 21:11:30.338724 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.337157    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.437786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.437758    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.652410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.650115    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.997465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:30.996487    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:31.235215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:31.235176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:31.298413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:31.298042    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:11:31.298413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:31.298120    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 21:11:31.324801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:31.324761    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:31.433410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:31.432043    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:32.303916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:32.303877    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:11:32.304233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:32.304197    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 21:11:32.797413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:32.790703    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:33.300440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:33.299931    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:11:33.300440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:33.300016    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 21:11:33.905349 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:33.905314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:34.298645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:34.298600    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:11:34.298863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:34.298682    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 21:11:34.683470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:34.678914    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:34.767997 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:34.767945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:35.004199 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:35.004094    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:35.713810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:35.713687    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:35.714487 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:35.714455    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:35.715857 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:35.715836    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:11:35.895311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:35.895269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:35.989666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:35.989572    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:36.022181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.022148    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"]
Oct 30 21:11:36.044060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.043746    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.044700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.044668    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"]
Oct 30 21:11:36.149414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.137728    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-f147de-dockercfg-479xs\""
Oct 30 21:11:36.149414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.137835    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-f147de\""
Oct 30 21:11:36.173421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.169266    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbzfr\" (UniqueName: \"kubernetes.io/projected/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-kube-api-access-dbzfr\") pod \"payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.173421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.169509    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.276115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.271342    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.276115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.271601    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbzfr\" (UniqueName: \"kubernetes.io/projected/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-kube-api-access-dbzfr\") pod \"payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.284114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.283053    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.309041 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.308966    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbzfr\" (UniqueName: \"kubernetes.io/projected/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-kube-api-access-dbzfr\") pod \"payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.367248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.367219    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.651460 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:11:36.651427    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311556e8_83a5_4fbe_a7ec_5efcb6d0432f.slice/crio-dcb42ca71a761cd546056d0209fa38c3083a498cb854cb87d872f0e52dd93695 WatchSource:0}: Error finding container dcb42ca71a761cd546056d0209fa38c3083a498cb854cb87d872f0e52dd93695: Status 404 returned error can't find the container with id dcb42ca71a761cd546056d0209fa38c3083a498cb854cb87d872f0e52dd93695
Oct 30 21:11:36.652251 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.652214    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"]
Oct 30 21:11:36.827993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.827952    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:36.970549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.970011    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2" event={"ID":"311556e8-83a5-4fbe-a7ec-5efcb6d0432f","Type":"ContainerStarted","Data":"c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c"}
Oct 30 21:11:36.970549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.970539    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2" event={"ID":"311556e8-83a5-4fbe-a7ec-5efcb6d0432f","Type":"ContainerStarted","Data":"dcb42ca71a761cd546056d0209fa38c3083a498cb854cb87d872f0e52dd93695"}
Oct 30 21:11:36.971597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.971163    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:36.989505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:36.989452    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2" podStartSLOduration=1.989437862 podStartE2EDuration="1.989437862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:36.986356479 +0000 UTC m=+2005.101090669" watchObservedRunningTime="2026-10-30 21:11:36.989437862 +0000 UTC m=+2005.104172051"
Oct 30 21:11:37.777606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:37.777570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:38.625628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:38.625595    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:39.386542 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:39.386511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:39.998197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:39.998156    4852 generic.go:356] "Generic (PLEG): container finished" podID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerID="1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f" exitCode=0
Oct 30 21:11:40.007765 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:40.007725    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" event={"ID":"5f57cfad-f8d4-4b39-a024-dfefeba9b431","Type":"ContainerDied","Data":"1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f"}
Oct 30 21:11:40.008259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:40.008227    4852 scope.go:122] "RemoveContainer" containerID="1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f"
Oct 30 21:11:40.010272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:40.010238    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:40.010367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:40.010314    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:40.010367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:40.010348    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:40.427277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:40.427084    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.008865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:41.008830    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" event={"ID":"5f57cfad-f8d4-4b39-a024-dfefeba9b431","Type":"ContainerStarted","Data":"9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6"}
Oct 30 21:11:41.009282 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:41.009262    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:11:41.010495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:41.010472    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 21:11:41.010591 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:41.010515    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 21:11:41.135643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:41.135594    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.176031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:41.175988    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:11:41.849914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:41.849876    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.009470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:42.009438    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 21:11:42.009651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:42.009507    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 21:11:42.775950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:42.775862    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:43.009503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:43.009466    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 21:11:43.010145 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:43.010097    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 21:11:43.629707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:43.629671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:44.010239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:44.010199    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 21:11:44.010674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:44.010267    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 21:11:44.420781 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:44.420699    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:45.009611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:45.009580    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 21:11:45.009788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:45.009647    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 21:11:45.331111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:45.331023    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:46.015870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:46.015834    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 21:11:46.016169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:46.016140    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 21:11:46.769523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:46.769482    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:47.171929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:47.171825    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:11:47.180675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:47.180642    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:11:47.709647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:47.709615    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.526667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:48.526625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.992024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:48.991947    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:11:49.331877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:49.331837    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:50.121311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:50.121266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:50.907816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:50.907776    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:51.583028 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:11:51.582991    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:11:51.668084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:51.666788    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:52.405836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:52.405804    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:53.072671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:53.072640    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:53.826568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:53.826532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:53.902698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:53.902651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:54.840759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:54.840723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:55.637436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:55.637399    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:55.856071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:55.855651    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": read tcp 10.133.0.2:39294->10.133.0.153:15021: read: connection reset by peer" start-of-body=
Oct 30 21:11:55.856071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:55.855729    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": read tcp 10.133.0.2:39294->10.133.0.153:15021: read: connection reset by peer"
Oct 30 21:11:56.249578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:56.249539    4852 generic.go:356] "Generic (PLEG): container finished" podID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerID="5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb" exitCode=0
Oct 30 21:11:56.249783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:56.249612    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" event={"ID":"60f52450-d4d3-4903-8be4-c97eac38e4f9","Type":"ContainerDied","Data":"5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb"}
Oct 30 21:11:56.250125 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:56.250109    4852 scope.go:122] "RemoveContainer" containerID="5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb"
Oct 30 21:11:56.252027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:56.252000    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:56.252119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:56.252071    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:56.252119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:56.252107    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:11:56.359784 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:56.359744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:57.125827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:57.125797    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:57.292309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:57.292275    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" event={"ID":"60f52450-d4d3-4903-8be4-c97eac38e4f9","Type":"ContainerStarted","Data":"fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9"}
Oct 30 21:11:57.920872 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:57.920837    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:58.130932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:58.130894    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:11:58.131333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:58.131214    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 21:11:58.131333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:58.131284    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 21:11:58.669412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:58.665891    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:59.132177 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:59.132143    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 21:11:59.132597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:59.132195    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 21:11:59.626272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:11:59.626229    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:00.131327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:00.131277    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 21:12:00.131586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:00.131342    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 21:12:00.321636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:00.321593    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:01.037170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:01.037138    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:01.131941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:01.131898    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 21:12:01.132128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:01.131958    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 21:12:02.132207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:02.132164    4852 patch_prober.go:29] interesting pod/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused" start-of-body=
Oct 30 21:12:02.132641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:02.132231    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.153:15021/healthz/ready\": dial tcp 10.133.0.153:15021: connect: connection refused"
Oct 30 21:12:02.242762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:02.242710    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:02.524375 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:02.524337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:03.136499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:03.136465    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:12:03.136989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:03.136968    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:12:03.137692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:03.137667    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:12:03.219130 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:03.219037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.024147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:04.024105    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.568664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:04.568631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.728940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:04.728909    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:05.100617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:05.100579    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:05.196746 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:05.196715    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:05.876414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:05.876369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:06.629530 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:06.629498    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:07.330784 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:07.330752    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:08.041965 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:08.041932    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:08.751573 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:08.751528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:09.534641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:09.534611    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:10.293678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:10.293638    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:11.006870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:11.006838    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:11.803876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:11.803833    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:12.627462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:12.627424    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:13.720856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:13.720824    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:14.735236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:14.735204    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:15.328340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:15.328305    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:15.598121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:15.598040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:16.509715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:16.509665    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:16.615858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:16.615823    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:16.788266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:16.788178    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.091813 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:17.091725    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.312277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:17.312241    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.526063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:17.526023    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.777277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:17.777187    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.957259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:17.957225    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:18.056952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:18.056639    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:18.772295 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:18.772253    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:18.880695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:18.880656    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:19.118168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:19.118082    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:19.387858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:19.387771    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:20.072881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:20.072832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:20.836769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:20.836733    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:21.560249 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:12:21.560214    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:12:21.829523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:21.829378    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:22.626484 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:22.626451    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:23.442516 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:23.442481    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:24.154857 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:24.154818    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:24.946532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:24.946495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:25.806069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:25.806022    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.045773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:26.045742    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.146306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:26.146213    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.246517 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:26.246481    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.648359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:26.648323    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:27.234407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:27.234366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:27.329459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:27.329377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:27.519410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:27.516956    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:27.972165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:27.972076    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:28.088825 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:28.088791    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:28.240022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:28.239928    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:28.347634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:28.347604    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:28.462690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:28.462643    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:28.670426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:28.669944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:29.267073 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:29.267036    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:29.368586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:29.368555    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:29.491368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:29.491330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:30.022303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:30.022260    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:30.755271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:30.755221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:31.639339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:31.639301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:31.800147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:31.800114    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:31.976259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:31.976166    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.074438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:32.074405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.175602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:32.175566    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.700870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:32.700821    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.800573 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:32.800542    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:33.124735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:33.124701    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:33.233642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:33.233551    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:33.351684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:33.351647    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:34.859202 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:34.859167    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:34.967842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:34.967803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:35.689204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:35.689167    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:36.492643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:36.492600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:36.590167 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:36.590131    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:37.788615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:37.788576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:38.129161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:38.129062    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:38.205768 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:38.205732    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:39.517803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:39.517768    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:39.615716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:39.615671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:39.854352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:39.854270    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:40.089060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:40.088985    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:40.336682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:40.336644    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:40.635273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:40.635074    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:40.735069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:40.735031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:41.443761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:41.443723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:42.225142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:42.225106    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:43.104275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:43.104245    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:43.806337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:43.806301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:44.692735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:44.692698    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:44.793155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:44.793124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:45.846276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:45.846244    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:46.634631 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:46.634599    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:47.574332 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:47.574292    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:47.699030 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:47.698996    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:48.447775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:48.447739    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:49.249249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:49.249218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:50.043077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:50.043037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:50.791559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:50.791527    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:51.560880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:51.560840    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:51.560880 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:12:51.560863    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:12:52.344763 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:52.344725    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:53.294814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:53.294777    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:53.415959 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:53.415925    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:53.521918 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:53.521885    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:54.617871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:54.617835    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:55.749558 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:55.749527    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:56.631903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:56.631859    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:57.638039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:57.637989    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:58.595671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:58.595619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:59.647080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.647042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:59.677239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.677187    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"]
Oct 30 21:12:59.682895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.682865    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.688863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.688831    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"]
Oct 30 21:12:59.689450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.689427    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-00909f-openshift-default-dockercfg-t87br\""
Oct 30 21:12:59.809823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.809787    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969d8145-ba12-4e33-837e-5f13d0d71291-istio-podinfo\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.809823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.809828    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-certs\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.810039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.809902    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-socket\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.810081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.810050    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969d8145-ba12-4e33-837e-5f13d0d71291-istiod-ca-cert\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.810081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.810075    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-istio-token\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.810156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.810108    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-credential-socket\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.810249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.810224    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-data\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.810286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.810260    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmqgg\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-kube-api-access-dmqgg\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.810329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.810289    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-envoy\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.911665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.911565    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-credential-socket\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.911665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.911638    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-data\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.911906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.911670    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqgg\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-kube-api-access-dmqgg\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.911906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.911704    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-envoy\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.911906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.911791    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969d8145-ba12-4e33-837e-5f13d0d71291-istio-podinfo\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.911906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.911818    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-certs\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.911906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.911869    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-socket\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.912165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.912102    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-credential-socket\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.912165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.912109    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969d8145-ba12-4e33-837e-5f13d0d71291-istiod-ca-cert\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.912280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.912192    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-istio-token\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.912370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.912340    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-certs\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.912538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.912436    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-socket\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.912719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.912692    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-data\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.912928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.912905    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969d8145-ba12-4e33-837e-5f13d0d71291-istiod-ca-cert\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.914085 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.914064    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-envoy\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.914548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.914525    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969d8145-ba12-4e33-837e-5f13d0d71291-istio-podinfo\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.919785 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.919759    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmqgg\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-kube-api-access-dmqgg\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:12:59.919896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:12:59.919770    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-istio-token\") pod \"e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") " pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:13:00.012433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:00.012367    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:13:00.304353 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:13:00.304320    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969d8145_ba12_4e33_837e_5f13d0d71291.slice/crio-e5e6c1d0b223df369367446e85db60cd4212363ce8cdda22420f4a0b318b7820 WatchSource:0}: Error finding container e5e6c1d0b223df369367446e85db60cd4212363ce8cdda22420f4a0b318b7820: Status 404 returned error can't find the container with id e5e6c1d0b223df369367446e85db60cd4212363ce8cdda22420f4a0b318b7820
Oct 30 21:13:00.304739 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:00.304713    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"]
Oct 30 21:13:00.307494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:00.307459    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:13:00.307616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:00.307530    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:13:00.307616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:00.307561    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:13:00.345768 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:00.345737    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" event={"ID":"969d8145-ba12-4e33-837e-5f13d0d71291","Type":"ContainerStarted","Data":"e5e6c1d0b223df369367446e85db60cd4212363ce8cdda22420f4a0b318b7820"}
Oct 30 21:13:00.537786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:00.537743    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:01.321692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:01.321652    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:01.415339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:01.413026    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" event={"ID":"969d8145-ba12-4e33-837e-5f13d0d71291","Type":"ContainerStarted","Data":"f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0"}
Oct 30 21:13:01.438565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:01.438508    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podStartSLOduration=2.438492744 podStartE2EDuration="2.438492744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:01.434963978 +0000 UTC m=+2089.549698167" watchObservedRunningTime="2026-10-30 21:13:01.438492744 +0000 UTC m=+2089.553226933"
Oct 30 21:13:02.013309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:02.013275    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:13:02.014745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:02.014723    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 21:13:02.014853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:02.014787    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 21:13:02.128397 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:02.128362    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:02.919659 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:02.919626    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:03.013160 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:03.013117    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 21:13:03.013488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:03.013214    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 21:13:03.640914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:03.640877    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:03.741069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:03.741029    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:04.013589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:04.013537    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 21:13:04.014096 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:04.013615    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 21:13:04.438171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:04.438072    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:04.640147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:04.640111    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:05.012974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:05.012939    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 21:13:05.013174 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:05.013007    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 21:13:05.183411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:05.182884    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:05.272150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:05.272025    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:05.789922 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:05.789891    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:06.013508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:06.013474    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 21:13:06.013722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:06.013532    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 21:13:07.012888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:07.012846    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 21:13:07.013302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:07.012929    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 21:13:07.510526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:07.506486    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:08.018554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:08.018522    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:13:08.018954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:08.018788    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:13:08.019654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:08.019629    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:13:08.432850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:08.432762    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:09.201316 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:09.201272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:09.301702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:09.301664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:09.420074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:09.415428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:10.246457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:10.246422    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:10.375881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:10.375845    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:10.492334 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:10.492297    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:10.696804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:10.696773    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:10.928495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:10.928455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:11.328690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:11.328651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:11.443623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:11.443569    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:11.692845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:11.692765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:11.989601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:11.989509    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:12.128497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:12.128462    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:13.360963 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:13.360877    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:14.056816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:14.056783    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:14.158454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:14.158418    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:14.268058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:14.267999    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:15.242234 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:15.242193    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:16.191668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:16.191626    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:17.108606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:17.108570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:18.215378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:18.214271    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:18.839301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:18.839263    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:18.961426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:18.961373    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:19.095452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:19.095343    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:19.886414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:19.881984    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:20.931936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:20.931899    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:21.561851 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:13:21.561784    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:13:21.955266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:21.955191    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.066974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:22.066940    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.238682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:22.238598    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.441055 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:22.441021    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.805305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:22.805273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.930875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:22.930836    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:23.194290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:23.194183    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:23.422083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:23.422048    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:23.523952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:23.523905    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:24.186038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:24.185996    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:24.289593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:24.289552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:24.881246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:24.881189    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:25.740568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:25.740530    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:26.533001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:26.532963    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:27.343524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:27.343489    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:28.180629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:28.180595    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:29.116519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:29.111250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:29.980021 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:29.979983    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:30.832306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:30.832268    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:31.716644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:31.716607    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:32.479588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:32.479549    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:33.345630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:33.345588    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:34.147163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:34.147130    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:34.969826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:34.969770    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:35.747852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:35.747809    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:36.654168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:36.654133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:37.598743 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:37.598678    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:38.724237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:38.724186    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:39.558253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:39.558212    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:40.442278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:40.442243    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:41.235537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:41.235490    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:42.090207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:42.090170    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:42.995740 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:42.995704    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:43.853305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:43.853268    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:44.757084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:44.757042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:45.560816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:45.560765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:46.431492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:46.431458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:47.243159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:47.243123    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:48.013533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:48.013489    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:48.846078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:48.846042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:49.573056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:49.573005    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:50.398525 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:50.398489    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:51.297546 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:51.297507    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:51.559911 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:13:51.559830    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:13:52.137157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:52.137055    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:52.884593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:52.884530    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:53.519207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:53.519175    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:53.685934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:53.685897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:54.006514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:54.006481    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:54.992304 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:54.992255    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:55.764930 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:55.764897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:56.551005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:56.550972    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:57.510596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:57.510562    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:58.249788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:58.249736    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:59.218312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:13:59.218255    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:00.057925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:00.057885    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:00.858704 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:00.858667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:01.675371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:01.675340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:02.563906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:02.563867    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:03.445869 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:03.445836    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:04.229020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:04.228982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:04.693460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:04.693427    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:05.163851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:05.163817    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:05.277805 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:05.277762    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:05.858373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:05.858314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:06.649313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:06.649282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:07.584065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:07.584034    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:08.358801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:08.358754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:08.847916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:08.847884    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:08.961467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:08.961431    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:09.680888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:09.680857    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:09.782915 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:09.782868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:11.089506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:11.089472    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:11.833753 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:11.833720    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:12.669286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:12.669248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:13.182828 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:13.182759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:13.295133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:13.295099    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:13.410511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:13.410449    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:13.681353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:13.681314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:14.522940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:14.522904    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:15.388862 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:15.388826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:16.519971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:16.519934    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:17.257221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:17.257173    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:18.035034 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:18.034991    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:18.961573 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:18.961533    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:19.771158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:19.771122    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:20.679952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:20.679903    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:21.207211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:21.207168    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:21.336592 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:21.336555    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:21.491594 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:21.491512    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:21.559434 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:14:21.559367    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:14:21.693222 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:21.693179    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:21.849245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:21.849215    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:21.982964 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:21.982923    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:22.125727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:22.125636    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:22.828638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:22.828605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:22.987571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:22.987528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:23.350099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:23.350061    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:23.833347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:23.833307    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:24.581783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:24.581740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:25.709562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:25.709523    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:26.674413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:26.674038    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:27.433303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:27.433269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:28.335078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:28.335040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:29.171411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:29.161881    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:30.020620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:30.020582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:30.753791 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:30.753757    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:31.548716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:31.548671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:32.008723 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:32.008686    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:32.134532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:32.134488    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:32.433581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:32.432012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:32.709193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:32.709097    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:32.827451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:32.827412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:33.052924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:33.052886    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:33.266159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:33.266116    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:33.410472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:33.409428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:33.572538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:33.572499    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:34.115067 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:34.115031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:34.243838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:34.243805    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:35.156423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.155063    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:35.258792 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.258758    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:35.531879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.531840    4852 generic.go:356] "Generic (PLEG): container finished" podID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerID="075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e" exitCode=0
Oct 30 21:14:35.532077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.531922    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" event={"ID":"618802d8-a3bc-4e09-846a-b3fd0ca1327c","Type":"ContainerDied","Data":"075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e"}
Oct 30 21:14:35.532571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.532548    4852 scope.go:122] "RemoveContainer" containerID="075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e"
Oct 30 21:14:35.538075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.538036    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:14:35.538221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.538135    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:14:35.538221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:35.538185    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:14:36.241292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:36.241236    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:36.543203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:36.543166    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" event={"ID":"618802d8-a3bc-4e09-846a-b3fd0ca1327c","Type":"ContainerStarted","Data":"fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c"}
Oct 30 21:14:36.874729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:36.874621    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:14:36.876512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:36.876482    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:14:36.876669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:36.876563    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 21:14:37.021184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:37.021153    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:37.874906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:37.874866    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:14:37.875322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:37.874930    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 21:14:37.995241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:37.995208    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:14:38.043871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:38.043811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:38.874409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:38.873850    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:14:38.874409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:38.873919    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 21:14:39.174993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:39.174882    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:39.875908 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:39.875854    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:14:39.876118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:39.875925    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 21:14:40.076009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:40.075964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:40.836678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:40.836646    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:40.877926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:40.877881    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:14:40.878253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:40.878213    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 21:14:41.442269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:41.442237    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:41.877420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:41.876770    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused" start-of-body=
Oct 30 21:14:41.877420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:41.876836    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.142:15021/healthz/ready\": dial tcp 10.133.0.142:15021: connect: connection refused"
Oct 30 21:14:42.133286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:42.133198    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:42.878072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:42.877977    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:14:42.879209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:42.879182    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:14:43.145001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:43.144894    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:43.934536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:43.934496    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:44.666264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:44.666224    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:45.639884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:45.639853    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:46.473223 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:46.473184    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:47.343088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:47.343051    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:48.253750 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:48.253719    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:49.173118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:49.173084    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:49.870576 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:49.870540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:50.667882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:50.667847    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:51.456185 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:51.456135    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:51.562172 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:14:51.562132    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:14:52.327150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:52.327105    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:53.102728 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:53.102693    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:53.888253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:53.888218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:54.285267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:54.285231    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:54.654258 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:54.654142    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:55.524184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:55.524148    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:56.439266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:56.439228    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:57.336507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:57.336472    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:58.189444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:58.189350    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:59.059227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:59.059185    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:59.820841 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:14:59.820809    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:00.212470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.212374    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"]
Oct 30 21:15:00.219332 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.219301    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"]
Oct 30 21:15:00.219677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.219500    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.224980 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.224956    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.244233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.244003    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"]
Oct 30 21:15:00.249315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.249287    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"]
Oct 30 21:15:00.277207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.277175    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"]
Oct 30 21:15:00.283187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.283158    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.284764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.284737    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 21:15:00.288322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.286623    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"]
Oct 30 21:15:00.291322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.291294    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-926rz\" (UniqueName: \"kubernetes.io/projected/779bd1e9-c4ac-4d54-bbbe-917526c270ac-kube-api-access-926rz\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.291485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.291374    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13a424a-d872-4016-82de-9d49f62e9998-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.291485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.291463    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/779bd1e9-c4ac-4d54-bbbe-917526c270ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.291632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.291606    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpj74\" (UniqueName: \"kubernetes.io/projected/f13a424a-d872-4016-82de-9d49f62e9998-kube-api-access-hpj74\") pod \"maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.292505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.292482    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.306409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.304151    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"]
Oct 30 21:15:00.310228 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.310165    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"]
Oct 30 21:15:00.310531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.310503    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.315834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.315808    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.321478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.321449    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"]
Oct 30 21:15:00.328876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.328843    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"]
Oct 30 21:15:00.346451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.343492    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"]
Oct 30 21:15:00.346451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.343593    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.346451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.345371    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"]
Oct 30 21:15:00.371199 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.371153    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"]
Oct 30 21:15:00.388460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.388431    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"]
Oct 30 21:15:00.394114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394077    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxfv\" (UniqueName: \"kubernetes.io/projected/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-kube-api-access-hdxfv\") pod \"maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.394285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394149    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-926rz\" (UniqueName: \"kubernetes.io/projected/779bd1e9-c4ac-4d54-bbbe-917526c270ac-kube-api-access-926rz\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.394285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394248    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13a424a-d872-4016-82de-9d49f62e9998-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.394430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394314    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/779bd1e9-c4ac-4d54-bbbe-917526c270ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.394430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394363    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qsmv\" (UniqueName: \"kubernetes.io/projected/cb636f54-64f0-4fe4-86c3-79864ec7a709-kube-api-access-2qsmv\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.394430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394411    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/135c52c6-069b-474f-996f-d463154006b0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.394606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394571    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpj74\" (UniqueName: \"kubernetes.io/projected/f13a424a-d872-4016-82de-9d49f62e9998-kube-api-access-hpj74\") pod \"maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.394673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394617    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7188c90b-a928-4434-a988-be7edecbdd9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.394673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394660    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.394834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394761    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8jfv\" (UniqueName: \"kubernetes.io/projected/7188c90b-a928-4434-a988-be7edecbdd9d-kube-api-access-l8jfv\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.394914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394852    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vktrd\" (UniqueName: \"kubernetes.io/projected/135c52c6-069b-474f-996f-d463154006b0-kube-api-access-vktrd\") pod \"maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.394914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.394904    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb636f54-64f0-4fe4-86c3-79864ec7a709-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.395221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.395193    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13a424a-d872-4016-82de-9d49f62e9998-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.395338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.395260    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/779bd1e9-c4ac-4d54-bbbe-917526c270ac-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.426106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.420853    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"]
Oct 30 21:15:00.429281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.429255    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.431219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.431198    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"]
Oct 30 21:15:00.442023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.441991    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpj74\" (UniqueName: \"kubernetes.io/projected/f13a424a-d872-4016-82de-9d49f62e9998-kube-api-access-hpj74\") pod \"maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.446323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.446297    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-926rz\" (UniqueName: \"kubernetes.io/projected/779bd1e9-c4ac-4d54-bbbe-917526c270ac-kube-api-access-926rz\") pod \"maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.446897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.446870    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"]
Oct 30 21:15:00.447079 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.447069    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.451165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.451140    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"]
Oct 30 21:15:00.495994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.495961    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vktrd\" (UniqueName: \"kubernetes.io/projected/135c52c6-069b-474f-996f-d463154006b0-kube-api-access-vktrd\") pod \"maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.496152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496020    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb636f54-64f0-4fe4-86c3-79864ec7a709-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.496152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496060    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.496152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496116    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t7x6\" (UniqueName: \"kubernetes.io/projected/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-kube-api-access-9t7x6\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.496323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496253    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxfv\" (UniqueName: \"kubernetes.io/projected/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-kube-api-access-hdxfv\") pod \"maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.496323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496287    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.496471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496432    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjck9\" (UniqueName: \"kubernetes.io/projected/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-kube-api-access-xjck9\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.496527 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496514    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qsmv\" (UniqueName: \"kubernetes.io/projected/cb636f54-64f0-4fe4-86c3-79864ec7a709-kube-api-access-2qsmv\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.496563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496537    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/135c52c6-069b-474f-996f-d463154006b0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.496632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496620    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7188c90b-a928-4434-a988-be7edecbdd9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.496672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496649    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.496761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496745    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8jfv\" (UniqueName: \"kubernetes.io/projected/7188c90b-a928-4434-a988-be7edecbdd9d-kube-api-access-l8jfv\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.496807 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.496782    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb636f54-64f0-4fe4-86c3-79864ec7a709-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.497197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.497177    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/135c52c6-069b-474f-996f-d463154006b0-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.497317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.497300    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.497363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.497345    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7188c90b-a928-4434-a988-be7edecbdd9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.507037 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.507012    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vktrd\" (UniqueName: \"kubernetes.io/projected/135c52c6-069b-474f-996f-d463154006b0-kube-api-access-vktrd\") pod \"maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.507163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.507039    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8jfv\" (UniqueName: \"kubernetes.io/projected/7188c90b-a928-4434-a988-be7edecbdd9d-kube-api-access-l8jfv\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.507163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.507083    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qsmv\" (UniqueName: \"kubernetes.io/projected/cb636f54-64f0-4fe4-86c3-79864ec7a709-kube-api-access-2qsmv\") pod \"maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.507163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.507118    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxfv\" (UniqueName: \"kubernetes.io/projected/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-kube-api-access-hdxfv\") pod \"maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.554335 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.554311    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:00.583840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.583812    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:00.597976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.597942    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.598118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.598073    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.598192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.598134    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t7x6\" (UniqueName: \"kubernetes.io/projected/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-kube-api-access-9t7x6\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.598260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.598211    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.598317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.598304    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjck9\" (UniqueName: \"kubernetes.io/projected/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-kube-api-access-xjck9\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.598433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.598374    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p95lp\" (UniqueName: \"kubernetes.io/projected/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-kube-api-access-p95lp\") pod \"maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.598871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.598853    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.598965 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.598943    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.608489 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.608455    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjck9\" (UniqueName: \"kubernetes.io/projected/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-kube-api-access-xjck9\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.608663 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.608635    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t7x6\" (UniqueName: \"kubernetes.io/projected/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-kube-api-access-9t7x6\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.624746 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.618246    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:00.646041 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.646005    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:00.658363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.655900    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:00.687762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.681695    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:00.699414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.698780    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:00.702412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.700604    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p95lp\" (UniqueName: \"kubernetes.io/projected/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-kube-api-access-p95lp\") pod \"maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.702412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.700843    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.705172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.705110    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:00.706617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.706565    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.732267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.732114    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p95lp\" (UniqueName: \"kubernetes.io/projected/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-kube-api-access-p95lp\") pod \"maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.789553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.789503    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:00.796998 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.796966    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:00.871800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:00.869805    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"]
Oct 30 21:15:01.100803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.100769    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"]
Oct 30 21:15:01.153105 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:01.153069    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13a424a_d872_4016_82de_9d49f62e9998.slice/crio-12a310a17ee2c0e140f13a8e9544ae473e495bac1433858519ce4e1094d4f243 WatchSource:0}: Error finding container 12a310a17ee2c0e140f13a8e9544ae473e495bac1433858519ce4e1094d4f243: Status 404 returned error can't find the container with id 12a310a17ee2c0e140f13a8e9544ae473e495bac1433858519ce4e1094d4f243
Oct 30 21:15:01.186961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.186929    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"]
Oct 30 21:15:01.345581 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:01.345545    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod135c52c6_069b_474f_996f_d463154006b0.slice/crio-368610f0f5378ef7b5f208f1a4c3f2183dc9b72d89a603cb68a6f023433a4820 WatchSource:0}: Error finding container 368610f0f5378ef7b5f208f1a4c3f2183dc9b72d89a603cb68a6f023433a4820: Status 404 returned error can't find the container with id 368610f0f5378ef7b5f208f1a4c3f2183dc9b72d89a603cb68a6f023433a4820
Oct 30 21:15:01.346450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.346417    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"]
Oct 30 21:15:01.348173 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:01.348150    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c5bd5e_97a9_4533_b149_2ce8c9a3c4f3.slice/crio-59367ae0aaead65f36563db8335da0b64228e5a89e74dbff82ec97015a6fe37b WatchSource:0}: Error finding container 59367ae0aaead65f36563db8335da0b64228e5a89e74dbff82ec97015a6fe37b: Status 404 returned error can't find the container with id 59367ae0aaead65f36563db8335da0b64228e5a89e74dbff82ec97015a6fe37b
Oct 30 21:15:01.359306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.359278    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"]
Oct 30 21:15:01.390144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.390122    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"]
Oct 30 21:15:01.394667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.394636    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"]
Oct 30 21:15:01.428247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.428224    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"]
Oct 30 21:15:01.432338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.432317    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"]
Oct 30 21:15:01.462673 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:01.458625    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d888772_04e0_4d87_8b74_fe08bcfd2a9d.slice/crio-9ab785cf599ee82d9e5cea238eba8a1d295502336044cf63ffa0f12b3368feeb WatchSource:0}: Error finding container 9ab785cf599ee82d9e5cea238eba8a1d295502336044cf63ffa0f12b3368feeb: Status 404 returned error can't find the container with id 9ab785cf599ee82d9e5cea238eba8a1d295502336044cf63ffa0f12b3368feeb
Oct 30 21:15:01.552225 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:01.552184    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f3a3e8_2522_4ec2_b6b9_b21d4c4c95ee.slice/crio-dc077352d2e261f25ca889ceb51264ffc5159a7ac0d592934795048c3a474220 WatchSource:0}: Error finding container dc077352d2e261f25ca889ceb51264ffc5159a7ac0d592934795048c3a474220: Status 404 returned error can't find the container with id dc077352d2e261f25ca889ceb51264ffc5159a7ac0d592934795048c3a474220
Oct 30 21:15:01.554302 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:01.554275    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8ca4b3_07e9_485d_bd38_c74d1e7ad1a6.slice/crio-a517540ae44f77b8f14a6da2207cfc2fa744147aa769b6b1a2e4325867f155db WatchSource:0}: Error finding container a517540ae44f77b8f14a6da2207cfc2fa744147aa769b6b1a2e4325867f155db: Status 404 returned error can't find the container with id a517540ae44f77b8f14a6da2207cfc2fa744147aa769b6b1a2e4325867f155db
Oct 30 21:15:01.586191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.586162    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:01.818299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.818197    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" event={"ID":"f13a424a-d872-4016-82de-9d49f62e9998","Type":"ContainerStarted","Data":"b925dba4d3c0e1501ba2ff46a7ec7cff1c5054a0fd1a2dd58d7857cbe3baef23"}
Oct 30 21:15:01.818299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.818257    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" event={"ID":"f13a424a-d872-4016-82de-9d49f62e9998","Type":"ContainerStarted","Data":"12a310a17ee2c0e140f13a8e9544ae473e495bac1433858519ce4e1094d4f243"}
Oct 30 21:15:01.819819 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.819786    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" event={"ID":"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3","Type":"ContainerStarted","Data":"59367ae0aaead65f36563db8335da0b64228e5a89e74dbff82ec97015a6fe37b"}
Oct 30 21:15:01.821931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.821859    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerStarted","Data":"7acac02fbedcd51c37511078f34d79494300feece1107986505414ddb8fe7616"}
Oct 30 21:15:01.821931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.821907    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerStarted","Data":"d87c30852cd8d0f6958dda3361e891875562a2443dd4729eabd4f8e348ce7504"}
Oct 30 21:15:01.823776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.823750    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" event={"ID":"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee","Type":"ContainerStarted","Data":"dc077352d2e261f25ca889ceb51264ffc5159a7ac0d592934795048c3a474220"}
Oct 30 21:15:01.825210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.825188    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerStarted","Data":"9ab785cf599ee82d9e5cea238eba8a1d295502336044cf63ffa0f12b3368feeb"}
Oct 30 21:15:01.826606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.826583    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" event={"ID":"135c52c6-069b-474f-996f-d463154006b0","Type":"ContainerStarted","Data":"368610f0f5378ef7b5f208f1a4c3f2183dc9b72d89a603cb68a6f023433a4820"}
Oct 30 21:15:01.827781 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.827762    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" event={"ID":"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6","Type":"ContainerStarted","Data":"a517540ae44f77b8f14a6da2207cfc2fa744147aa769b6b1a2e4325867f155db"}
Oct 30 21:15:01.829338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.829319    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerStarted","Data":"ae793ce56ef855611ced1d0a1c9f4a178106290e24528db6fead8398f1f1f5b0"}
Oct 30 21:15:01.829459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.829359    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerStarted","Data":"cb0dfc7dc7736180f90289b9e7c78a4baf2eaf7a162c09cb1aeb41787d960e17"}
Oct 30 21:15:01.830801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.830781    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" event={"ID":"cb636f54-64f0-4fe4-86c3-79864ec7a709","Type":"ContainerStarted","Data":"4e77879fd8afa479e08c92bbe500930f4c2bf008d8adcd9ebfd7ef4e36dec48f"}
Oct 30 21:15:01.841524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.841478    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" podStartSLOduration=1.841465451 podStartE2EDuration="1.841465451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:01.836209973 +0000 UTC m=+2209.950944163" watchObservedRunningTime="2026-10-30 21:15:01.841465451 +0000 UTC m=+2209.956199638"
Oct 30 21:15:01.858602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.858550    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" podStartSLOduration=1.85853632 podStartE2EDuration="1.85853632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:01.855135515 +0000 UTC m=+2209.969869706" watchObservedRunningTime="2026-10-30 21:15:01.85853632 +0000 UTC m=+2209.973270512"
Oct 30 21:15:01.877526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:01.877480    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" podStartSLOduration=1.877465151 podStartE2EDuration="1.877465151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:01.871660184 +0000 UTC m=+2209.986394374" watchObservedRunningTime="2026-10-30 21:15:01.877465151 +0000 UTC m=+2209.992199341"
Oct 30 21:15:02.532998 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.532962    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:02.840903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.840869    4852 generic.go:356] "Generic (PLEG): container finished" podID="135c52c6-069b-474f-996f-d463154006b0" containerID="3173c7bd9dbb0aff0c40227dc59000bbe8a2467dbaf34604b67bc4d2c52b89bf" exitCode=60
Oct 30 21:15:02.841121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.840967    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" event={"ID":"135c52c6-069b-474f-996f-d463154006b0","Type":"ContainerDied","Data":"3173c7bd9dbb0aff0c40227dc59000bbe8a2467dbaf34604b67bc4d2c52b89bf"}
Oct 30 21:15:02.841327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.841298    4852 scope.go:122] "RemoveContainer" containerID="3173c7bd9dbb0aff0c40227dc59000bbe8a2467dbaf34604b67bc4d2c52b89bf"
Oct 30 21:15:02.842832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.842806    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" event={"ID":"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6","Type":"ContainerStarted","Data":"91b2d2615c438225fa1a51fa0bab4fa125f9ceb4ad5a40cc5ec87a5eb33c5a0d"}
Oct 30 21:15:02.845216 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.845188    4852 generic.go:356] "Generic (PLEG): container finished" podID="7188c90b-a928-4434-a988-be7edecbdd9d" containerID="ae793ce56ef855611ced1d0a1c9f4a178106290e24528db6fead8398f1f1f5b0" exitCode=60
Oct 30 21:15:02.845337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.845218    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerDied","Data":"ae793ce56ef855611ced1d0a1c9f4a178106290e24528db6fead8398f1f1f5b0"}
Oct 30 21:15:02.845844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.845646    4852 scope.go:122] "RemoveContainer" containerID="ae793ce56ef855611ced1d0a1c9f4a178106290e24528db6fead8398f1f1f5b0"
Oct 30 21:15:02.847579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.847554    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" event={"ID":"cb636f54-64f0-4fe4-86c3-79864ec7a709","Type":"ContainerStarted","Data":"db2a2a9e8e202492f2bedcee49dd40ef2363c2ab37a2f669259027a7abddba1a"}
Oct 30 21:15:02.849507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.849482    4852 generic.go:356] "Generic (PLEG): container finished" podID="f13a424a-d872-4016-82de-9d49f62e9998" containerID="b925dba4d3c0e1501ba2ff46a7ec7cff1c5054a0fd1a2dd58d7857cbe3baef23" exitCode=60
Oct 30 21:15:02.849733 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.849569    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" event={"ID":"f13a424a-d872-4016-82de-9d49f62e9998","Type":"ContainerDied","Data":"b925dba4d3c0e1501ba2ff46a7ec7cff1c5054a0fd1a2dd58d7857cbe3baef23"}
Oct 30 21:15:02.849932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.849911    4852 scope.go:122] "RemoveContainer" containerID="b925dba4d3c0e1501ba2ff46a7ec7cff1c5054a0fd1a2dd58d7857cbe3baef23"
Oct 30 21:15:02.851823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.851799    4852 generic.go:356] "Generic (PLEG): container finished" podID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerID="94e3a671cf9db1619d9052eb146a9f2e66117127a8614376a791f17e3a481dce" exitCode=60
Oct 30 21:15:02.851921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.851877    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" event={"ID":"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3","Type":"ContainerDied","Data":"94e3a671cf9db1619d9052eb146a9f2e66117127a8614376a791f17e3a481dce"}
Oct 30 21:15:02.852478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.852130    4852 scope.go:122] "RemoveContainer" containerID="94e3a671cf9db1619d9052eb146a9f2e66117127a8614376a791f17e3a481dce"
Oct 30 21:15:02.854889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.854525    4852 generic.go:356] "Generic (PLEG): container finished" podID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerID="7acac02fbedcd51c37511078f34d79494300feece1107986505414ddb8fe7616" exitCode=60
Oct 30 21:15:02.854983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.854632    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerDied","Data":"7acac02fbedcd51c37511078f34d79494300feece1107986505414ddb8fe7616"}
Oct 30 21:15:02.855030 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.854864    4852 scope.go:122] "RemoveContainer" containerID="7acac02fbedcd51c37511078f34d79494300feece1107986505414ddb8fe7616"
Oct 30 21:15:02.865284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.865252    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" event={"ID":"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee","Type":"ContainerStarted","Data":"719916ee02d9541238370322ef0a33a76b3f92724680c61b8d77e4fece2be910"}
Oct 30 21:15:02.868043 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.868016    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerStarted","Data":"33041157b19f9231b7a7d6ba1585ee43cbdb18a6a0245ea321b6c82523c8e49d"}
Oct 30 21:15:02.868492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.868374    4852 scope.go:122] "RemoveContainer" containerID="33041157b19f9231b7a7d6ba1585ee43cbdb18a6a0245ea321b6c82523c8e49d"
Oct 30 21:15:02.903111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.903056    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" podStartSLOduration=2.903039771 podStartE2EDuration="2.903039771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:02.899786505 +0000 UTC m=+2211.014520697" watchObservedRunningTime="2026-10-30 21:15:02.903039771 +0000 UTC m=+2211.017774010"
Oct 30 21:15:02.977644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.977590    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" podStartSLOduration=2.977573473 podStartE2EDuration="2.977573473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:02.974892204 +0000 UTC m=+2211.089626385" watchObservedRunningTime="2026-10-30 21:15:02.977573473 +0000 UTC m=+2211.092307662"
Oct 30 21:15:02.992114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:02.992057    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" podStartSLOduration=2.992040653 podStartE2EDuration="2.992040653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:02.990658973 +0000 UTC m=+2211.105393163" watchObservedRunningTime="2026-10-30 21:15:02.992040653 +0000 UTC m=+2211.106774877"
Oct 30 21:15:03.359430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.359341    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:03.877516 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.877485    4852 generic.go:356] "Generic (PLEG): container finished" podID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerID="719916ee02d9541238370322ef0a33a76b3f92724680c61b8d77e4fece2be910" exitCode=60
Oct 30 21:15:03.877968 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.877575    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" event={"ID":"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee","Type":"ContainerDied","Data":"719916ee02d9541238370322ef0a33a76b3f92724680c61b8d77e4fece2be910"}
Oct 30 21:15:03.877968 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.877852    4852 scope.go:122] "RemoveContainer" containerID="719916ee02d9541238370322ef0a33a76b3f92724680c61b8d77e4fece2be910"
Oct 30 21:15:03.879827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.879721    4852 generic.go:356] "Generic (PLEG): container finished" podID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerID="33041157b19f9231b7a7d6ba1585ee43cbdb18a6a0245ea321b6c82523c8e49d" exitCode=60
Oct 30 21:15:03.879827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.879797    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerDied","Data":"33041157b19f9231b7a7d6ba1585ee43cbdb18a6a0245ea321b6c82523c8e49d"}
Oct 30 21:15:03.879980 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.879835    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerStarted","Data":"1595873a3771c311c5de2ec1b6265d7f5da3d4de60b297e66de7de7bcfc93c91"}
Oct 30 21:15:03.882788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.882756    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" event={"ID":"135c52c6-069b-474f-996f-d463154006b0","Type":"ContainerStarted","Data":"689f97f253fb703f0810459cbc75f902d694e3cb1cc9cef9c5d338c7f3d3a47b"}
Oct 30 21:15:03.884313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.884287    4852 generic.go:356] "Generic (PLEG): container finished" podID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerID="91b2d2615c438225fa1a51fa0bab4fa125f9ceb4ad5a40cc5ec87a5eb33c5a0d" exitCode=60
Oct 30 21:15:03.884447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.884366    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" event={"ID":"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6","Type":"ContainerDied","Data":"91b2d2615c438225fa1a51fa0bab4fa125f9ceb4ad5a40cc5ec87a5eb33c5a0d"}
Oct 30 21:15:03.884606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.884590    4852 scope.go:122] "RemoveContainer" containerID="91b2d2615c438225fa1a51fa0bab4fa125f9ceb4ad5a40cc5ec87a5eb33c5a0d"
Oct 30 21:15:03.886434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.886411    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerStarted","Data":"b6eebcec73a7ddecc1f8f0d7ac85f3a694df4bdd68f1a7c0188b557a100fc3d6"}
Oct 30 21:15:03.887906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.887884    4852 generic.go:356] "Generic (PLEG): container finished" podID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerID="db2a2a9e8e202492f2bedcee49dd40ef2363c2ab37a2f669259027a7abddba1a" exitCode=60
Oct 30 21:15:03.887989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.887966    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" event={"ID":"cb636f54-64f0-4fe4-86c3-79864ec7a709","Type":"ContainerDied","Data":"db2a2a9e8e202492f2bedcee49dd40ef2363c2ab37a2f669259027a7abddba1a"}
Oct 30 21:15:03.888144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.888129    4852 scope.go:122] "RemoveContainer" containerID="db2a2a9e8e202492f2bedcee49dd40ef2363c2ab37a2f669259027a7abddba1a"
Oct 30 21:15:03.890058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.890036    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" event={"ID":"f13a424a-d872-4016-82de-9d49f62e9998","Type":"ContainerStarted","Data":"1994a232f0102145507ed246482376172f0e008b106ff88ab5fdd294d988fb80"}
Oct 30 21:15:03.892312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.892287    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" event={"ID":"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3","Type":"ContainerStarted","Data":"826b70f11032ec77ff5f8041182b61cc671b843820fedf3a3bca437f23842563"}
Oct 30 21:15:03.896045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.896020    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerStarted","Data":"bf1e53490dea71d5db80ba42f894a1c1fa9bb8340eba9624c60f0830fa63cef7"}
Oct 30 21:15:03.953723 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.950265    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" podStartSLOduration=3.950245293 podStartE2EDuration="3.950245293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:03.948648265 +0000 UTC m=+2212.063382457" watchObservedRunningTime="2026-10-30 21:15:03.950245293 +0000 UTC m=+2212.064979484"
Oct 30 21:15:03.992527 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:03.992474    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" podStartSLOduration=3.992460858 podStartE2EDuration="3.992460858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:03.988772218 +0000 UTC m=+2212.103506409" watchObservedRunningTime="2026-10-30 21:15:03.992460858 +0000 UTC m=+2212.107195105"
Oct 30 21:15:04.013292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.013203    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" podStartSLOduration=4.013189667 podStartE2EDuration="4.013189667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:04.006981411 +0000 UTC m=+2212.121715603" watchObservedRunningTime="2026-10-30 21:15:04.013189667 +0000 UTC m=+2212.127923892"
Oct 30 21:15:04.168971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.168938    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:04.449857 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.449819    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:04.810001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.809964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:04.908309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.908212    4852 generic.go:356] "Generic (PLEG): container finished" podID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerID="bf1e53490dea71d5db80ba42f894a1c1fa9bb8340eba9624c60f0830fa63cef7" exitCode=60
Oct 30 21:15:04.908759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.908446    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerDied","Data":"bf1e53490dea71d5db80ba42f894a1c1fa9bb8340eba9624c60f0830fa63cef7"}
Oct 30 21:15:04.908759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.908566    4852 scope.go:122] "RemoveContainer" containerID="7acac02fbedcd51c37511078f34d79494300feece1107986505414ddb8fe7616"
Oct 30 21:15:04.908759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.908670    4852 scope.go:122] "RemoveContainer" containerID="bf1e53490dea71d5db80ba42f894a1c1fa9bb8340eba9624c60f0830fa63cef7"
Oct 30 21:15:04.909153 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:04.909128    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g_odh-ai-gateway-infra(779bd1e9-c4ac-4d54-bbbe-917526c270ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac"
Oct 30 21:15:04.911310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.911285    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" event={"ID":"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee","Type":"ContainerStarted","Data":"49904dcb58543d08e021cf593f869620cbcbdc60a32522e3bafb70ffd8881b35"}
Oct 30 21:15:04.913559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.913535    4852 generic.go:356] "Generic (PLEG): container finished" podID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerID="1595873a3771c311c5de2ec1b6265d7f5da3d4de60b297e66de7de7bcfc93c91" exitCode=60
Oct 30 21:15:04.913680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.913615    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerDied","Data":"1595873a3771c311c5de2ec1b6265d7f5da3d4de60b297e66de7de7bcfc93c91"}
Oct 30 21:15:04.913887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.913871    4852 scope.go:122] "RemoveContainer" containerID="1595873a3771c311c5de2ec1b6265d7f5da3d4de60b297e66de7de7bcfc93c91"
Oct 30 21:15:04.914219 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:04.914197    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk_odh-ai-gateway-infra(8d888772-04e0-4d87-8b74-fe08bcfd2a9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d"
Oct 30 21:15:04.915879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.915849    4852 generic.go:356] "Generic (PLEG): container finished" podID="135c52c6-069b-474f-996f-d463154006b0" containerID="689f97f253fb703f0810459cbc75f902d694e3cb1cc9cef9c5d338c7f3d3a47b" exitCode=60
Oct 30 21:15:04.915985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.915922    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" event={"ID":"135c52c6-069b-474f-996f-d463154006b0","Type":"ContainerDied","Data":"689f97f253fb703f0810459cbc75f902d694e3cb1cc9cef9c5d338c7f3d3a47b"}
Oct 30 21:15:04.916121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.916104    4852 scope.go:122] "RemoveContainer" containerID="689f97f253fb703f0810459cbc75f902d694e3cb1cc9cef9c5d338c7f3d3a47b"
Oct 30 21:15:04.916398 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:04.916367    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7_odh-ai-gateway-infra(135c52c6-069b-474f-996f-d463154006b0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" podUID="135c52c6-069b-474f-996f-d463154006b0"
Oct 30 21:15:04.917974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.917950    4852 generic.go:356] "Generic (PLEG): container finished" podID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerID="4e94cbca97c440808ca5d9bed44852b57c3db64d5c644cbfe76f6fe89b638267" exitCode=60
Oct 30 21:15:04.918210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.918007    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" event={"ID":"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6","Type":"ContainerDied","Data":"4e94cbca97c440808ca5d9bed44852b57c3db64d5c644cbfe76f6fe89b638267"}
Oct 30 21:15:04.918311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.918293    4852 scope.go:122] "RemoveContainer" containerID="4e94cbca97c440808ca5d9bed44852b57c3db64d5c644cbfe76f6fe89b638267"
Oct 30 21:15:04.918602 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:04.918578    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h_odh-ai-gateway-infra(8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6"
Oct 30 21:15:04.920328 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.920304    4852 generic.go:356] "Generic (PLEG): container finished" podID="7188c90b-a928-4434-a988-be7edecbdd9d" containerID="b6eebcec73a7ddecc1f8f0d7ac85f3a694df4bdd68f1a7c0188b557a100fc3d6" exitCode=60
Oct 30 21:15:04.920488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.920459    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerDied","Data":"b6eebcec73a7ddecc1f8f0d7ac85f3a694df4bdd68f1a7c0188b557a100fc3d6"}
Oct 30 21:15:04.920685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.920669    4852 scope.go:122] "RemoveContainer" containerID="b6eebcec73a7ddecc1f8f0d7ac85f3a694df4bdd68f1a7c0188b557a100fc3d6"
Oct 30 21:15:04.920970 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:04.920950    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk_odh-ai-gateway-infra(7188c90b-a928-4434-a988-be7edecbdd9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" podUID="7188c90b-a928-4434-a988-be7edecbdd9d"
Oct 30 21:15:04.922885 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.922864    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" event={"ID":"cb636f54-64f0-4fe4-86c3-79864ec7a709","Type":"ContainerStarted","Data":"4fb3057fba08799838c6cc586f9d30dd9de4fe5b135854166b3c8d4a4639bc9f"}
Oct 30 21:15:04.925962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.925939    4852 generic.go:356] "Generic (PLEG): container finished" podID="f13a424a-d872-4016-82de-9d49f62e9998" containerID="1994a232f0102145507ed246482376172f0e008b106ff88ab5fdd294d988fb80" exitCode=60
Oct 30 21:15:04.926061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.926025    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" event={"ID":"f13a424a-d872-4016-82de-9d49f62e9998","Type":"ContainerDied","Data":"1994a232f0102145507ed246482376172f0e008b106ff88ab5fdd294d988fb80"}
Oct 30 21:15:04.926254 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.926233    4852 scope.go:122] "RemoveContainer" containerID="1994a232f0102145507ed246482376172f0e008b106ff88ab5fdd294d988fb80"
Oct 30 21:15:04.926537 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:04.926509    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl_odh-ai-gateway-infra(f13a424a-d872-4016-82de-9d49f62e9998)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" podUID="f13a424a-d872-4016-82de-9d49f62e9998"
Oct 30 21:15:04.928785 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.928613    4852 generic.go:356] "Generic (PLEG): container finished" podID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerID="826b70f11032ec77ff5f8041182b61cc671b843820fedf3a3bca437f23842563" exitCode=60
Oct 30 21:15:04.928785 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.928694    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" event={"ID":"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3","Type":"ContainerDied","Data":"826b70f11032ec77ff5f8041182b61cc671b843820fedf3a3bca437f23842563"}
Oct 30 21:15:04.928785 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.928777    4852 scope.go:122] "RemoveContainer" containerID="33041157b19f9231b7a7d6ba1585ee43cbdb18a6a0245ea321b6c82523c8e49d"
Oct 30 21:15:04.928977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.928922    4852 scope.go:122] "RemoveContainer" containerID="826b70f11032ec77ff5f8041182b61cc671b843820fedf3a3bca437f23842563"
Oct 30 21:15:04.929218 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:04.929194    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7_odh-ai-gateway-infra(53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3"
Oct 30 21:15:04.944611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.944590    4852 scope.go:122] "RemoveContainer" containerID="3173c7bd9dbb0aff0c40227dc59000bbe8a2467dbaf34604b67bc4d2c52b89bf"
Oct 30 21:15:04.962292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.962267    4852 scope.go:122] "RemoveContainer" containerID="91b2d2615c438225fa1a51fa0bab4fa125f9ceb4ad5a40cc5ec87a5eb33c5a0d"
Oct 30 21:15:04.985019 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:04.984997    4852 scope.go:122] "RemoveContainer" containerID="ae793ce56ef855611ced1d0a1c9f4a178106290e24528db6fead8398f1f1f5b0"
Oct 30 21:15:05.002778 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.002405    4852 scope.go:122] "RemoveContainer" containerID="b925dba4d3c0e1501ba2ff46a7ec7cff1c5054a0fd1a2dd58d7857cbe3baef23"
Oct 30 21:15:05.018283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.018259    4852 scope.go:122] "RemoveContainer" containerID="94e3a671cf9db1619d9052eb146a9f2e66117127a8614376a791f17e3a481dce"
Oct 30 21:15:05.060630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.060600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:05.931374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.931344    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:05.938657 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.938615    4852 scope.go:122] "RemoveContainer" containerID="bf1e53490dea71d5db80ba42f894a1c1fa9bb8340eba9624c60f0830fa63cef7"
Oct 30 21:15:05.939035 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.939010    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g_odh-ai-gateway-infra(779bd1e9-c4ac-4d54-bbbe-917526c270ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac"
Oct 30 21:15:05.940261 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.940238    4852 generic.go:356] "Generic (PLEG): container finished" podID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerID="49904dcb58543d08e021cf593f869620cbcbdc60a32522e3bafb70ffd8881b35" exitCode=60
Oct 30 21:15:05.940377 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.940347    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" event={"ID":"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee","Type":"ContainerDied","Data":"49904dcb58543d08e021cf593f869620cbcbdc60a32522e3bafb70ffd8881b35"}
Oct 30 21:15:05.940463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.940446    4852 scope.go:122] "RemoveContainer" containerID="719916ee02d9541238370322ef0a33a76b3f92724680c61b8d77e4fece2be910"
Oct 30 21:15:05.940629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.940612    4852 scope.go:122] "RemoveContainer" containerID="49904dcb58543d08e021cf593f869620cbcbdc60a32522e3bafb70ffd8881b35"
Oct 30 21:15:05.940960 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.940939    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg_odh-ai-gateway-infra(94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee"
Oct 30 21:15:05.942422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.942404    4852 scope.go:122] "RemoveContainer" containerID="1595873a3771c311c5de2ec1b6265d7f5da3d4de60b297e66de7de7bcfc93c91"
Oct 30 21:15:05.942735 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.942714    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk_odh-ai-gateway-infra(8d888772-04e0-4d87-8b74-fe08bcfd2a9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d"
Oct 30 21:15:05.944104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.944085    4852 scope.go:122] "RemoveContainer" containerID="689f97f253fb703f0810459cbc75f902d694e3cb1cc9cef9c5d338c7f3d3a47b"
Oct 30 21:15:05.944447 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.944421    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7_odh-ai-gateway-infra(135c52c6-069b-474f-996f-d463154006b0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" podUID="135c52c6-069b-474f-996f-d463154006b0"
Oct 30 21:15:05.946093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.946074    4852 scope.go:122] "RemoveContainer" containerID="4e94cbca97c440808ca5d9bed44852b57c3db64d5c644cbfe76f6fe89b638267"
Oct 30 21:15:05.946322 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.946303    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h_odh-ai-gateway-infra(8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6"
Oct 30 21:15:05.948605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.948585    4852 scope.go:122] "RemoveContainer" containerID="b6eebcec73a7ddecc1f8f0d7ac85f3a694df4bdd68f1a7c0188b557a100fc3d6"
Oct 30 21:15:05.948867 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.948845    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk_odh-ai-gateway-infra(7188c90b-a928-4434-a988-be7edecbdd9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" podUID="7188c90b-a928-4434-a988-be7edecbdd9d"
Oct 30 21:15:05.950480 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.950449    4852 generic.go:356] "Generic (PLEG): container finished" podID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerID="4fb3057fba08799838c6cc586f9d30dd9de4fe5b135854166b3c8d4a4639bc9f" exitCode=60
Oct 30 21:15:05.950589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.950481    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" event={"ID":"cb636f54-64f0-4fe4-86c3-79864ec7a709","Type":"ContainerDied","Data":"4fb3057fba08799838c6cc586f9d30dd9de4fe5b135854166b3c8d4a4639bc9f"}
Oct 30 21:15:05.950776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.950759    4852 scope.go:122] "RemoveContainer" containerID="4fb3057fba08799838c6cc586f9d30dd9de4fe5b135854166b3c8d4a4639bc9f"
Oct 30 21:15:05.951051 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.951028    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx_odh-ai-gateway-infra(cb636f54-64f0-4fe4-86c3-79864ec7a709)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709"
Oct 30 21:15:05.952503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.952483    4852 scope.go:122] "RemoveContainer" containerID="1994a232f0102145507ed246482376172f0e008b106ff88ab5fdd294d988fb80"
Oct 30 21:15:05.952768 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.952747    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl_odh-ai-gateway-infra(f13a424a-d872-4016-82de-9d49f62e9998)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" podUID="f13a424a-d872-4016-82de-9d49f62e9998"
Oct 30 21:15:05.954449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.954422    4852 scope.go:122] "RemoveContainer" containerID="826b70f11032ec77ff5f8041182b61cc671b843820fedf3a3bca437f23842563"
Oct 30 21:15:05.954709 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:05.954688    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7_odh-ai-gateway-infra(53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3"
Oct 30 21:15:05.956362 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:05.956347    4852 scope.go:122] "RemoveContainer" containerID="db2a2a9e8e202492f2bedcee49dd40ef2363c2ab37a2f669259027a7abddba1a"
Oct 30 21:15:06.338511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.338472    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"]
Oct 30 21:15:06.346276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.346246    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.397830 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.377611    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"]
Oct 30 21:15:06.521464 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521422    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.521679 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521653    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd2s7\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-kube-api-access-rd2s7\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.521722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521706    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.521790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521776    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.521838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521827    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.521880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521859    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.521934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521883    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.522009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.521998    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.522077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.522065    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623369    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2s7\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-kube-api-access-rd2s7\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623468    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623687 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623495    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623687 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623549    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623687 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623664    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623805 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623703    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623805 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623731    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623819    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.623966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.623920    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.624084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.624060    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.624163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.624084    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.624220 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.624158    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.624281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.624238    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.624371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.624344    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.626458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.626431    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.626704 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.626687    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.632771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.632744    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.632877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.632839    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd2s7\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-kube-api-access-rd2s7\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.682370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.682342    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:06.856811 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.856765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:06.965237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.965155    4852 scope.go:122] "RemoveContainer" containerID="4fb3057fba08799838c6cc586f9d30dd9de4fe5b135854166b3c8d4a4639bc9f"
Oct 30 21:15:06.965738 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:06.965642    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx_odh-ai-gateway-infra(cb636f54-64f0-4fe4-86c3-79864ec7a709)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709"
Oct 30 21:15:06.967453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.967431    4852 scope.go:122] "RemoveContainer" containerID="49904dcb58543d08e021cf593f869620cbcbdc60a32522e3bafb70ffd8881b35"
Oct 30 21:15:06.967815 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:06.967796    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg_odh-ai-gateway-infra(94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee"
Oct 30 21:15:06.984989 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:06.984953    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c32cf1_82b7_47b4_b01e_91f1d80192ac.slice/crio-28d9e874ecce35682ba522922a22771f90e30a9cd0acfd13bc788f472ba7cf4e WatchSource:0}: Error finding container 28d9e874ecce35682ba522922a22771f90e30a9cd0acfd13bc788f472ba7cf4e: Status 404 returned error can't find the container with id 28d9e874ecce35682ba522922a22771f90e30a9cd0acfd13bc788f472ba7cf4e
Oct 30 21:15:06.985616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.985587    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"]
Oct 30 21:15:06.988657 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.988624    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:06.988744 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.988695    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:06.988744 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:06.988724    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:07.679417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:07.678633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:07.985756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:07.985658    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" event={"ID":"e3c32cf1-82b7-47b4-b01e-91f1d80192ac","Type":"ContainerStarted","Data":"ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5"}
Oct 30 21:15:07.985756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:07.985717    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" event={"ID":"e3c32cf1-82b7-47b4-b01e-91f1d80192ac","Type":"ContainerStarted","Data":"28d9e874ecce35682ba522922a22771f90e30a9cd0acfd13bc788f472ba7cf4e"}
Oct 30 21:15:08.015824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:08.015773    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podStartSLOduration=2.015758838 podStartE2EDuration="2.015758838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:08.011966038 +0000 UTC m=+2216.126700228" watchObservedRunningTime="2026-10-30 21:15:08.015758838 +0000 UTC m=+2216.130493087"
Oct 30 21:15:08.637256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:08.637221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:08.691257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:08.690829    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:08.700412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:08.698512    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 21:15:08.700412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:08.698577    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 21:15:09.689410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:09.688874    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 21:15:09.689410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:09.688941    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 21:15:10.375707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:10.375667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:10.684456 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:10.682905    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 21:15:10.684456 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:10.682975    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 21:15:11.333203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:11.333170    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:11.683569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:11.683466    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 21:15:11.684196 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:11.683800    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 21:15:12.203118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:12.203081    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:12.414662 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:12.414622    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:12.696412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:12.695578    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 21:15:12.696412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:12.695640    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 21:15:13.779002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:13.778965    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:13.779924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:13.779903    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:13.780948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:13.780924    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:15:13.913113 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:13.913074    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:14.786172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:14.786137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:15.663839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:15.663800    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:15.997603 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:15.997514    4852 scope.go:122] "RemoveContainer" containerID="b6eebcec73a7ddecc1f8f0d7ac85f3a694df4bdd68f1a7c0188b557a100fc3d6"
Oct 30 21:15:15.998001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:15.997658    4852 scope.go:122] "RemoveContainer" containerID="bf1e53490dea71d5db80ba42f894a1c1fa9bb8340eba9624c60f0830fa63cef7"
Oct 30 21:15:16.457789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:16.457712    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:16.997873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:16.997838    4852 scope.go:122] "RemoveContainer" containerID="4e94cbca97c440808ca5d9bed44852b57c3db64d5c644cbfe76f6fe89b638267"
Oct 30 21:15:16.998371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:16.998287    4852 scope.go:122] "RemoveContainer" containerID="4fb3057fba08799838c6cc586f9d30dd9de4fe5b135854166b3c8d4a4639bc9f"
Oct 30 21:15:16.998503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:16.998480    4852 scope.go:122] "RemoveContainer" containerID="826b70f11032ec77ff5f8041182b61cc671b843820fedf3a3bca437f23842563"
Oct 30 21:15:17.073244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:17.073208    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerStarted","Data":"7dd62810f4f4907d5f55d09227f3efe0776244feec39fabb382050ffe6591544"}
Oct 30 21:15:17.073908 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:17.073883    4852 scope.go:122] "RemoveContainer" containerID="7dd62810f4f4907d5f55d09227f3efe0776244feec39fabb382050ffe6591544"
Oct 30 21:15:17.074422 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:17.074376    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk_odh-ai-gateway-infra(7188c90b-a928-4434-a988-be7edecbdd9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" podUID="7188c90b-a928-4434-a988-be7edecbdd9d"
Oct 30 21:15:17.075769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:17.075746    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerStarted","Data":"7c88fe54d34fa6c4b27ffd306f16e2b2f58e02078697fa809b2443e536f10c4b"}
Oct 30 21:15:17.220867 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:17.220824    4852 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779bd1e9_c4ac_4d54_bbbe_917526c270ac.slice/crio-conmon-7c88fe54d34fa6c4b27ffd306f16e2b2f58e02078697fa809b2443e536f10c4b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779bd1e9_c4ac_4d54_bbbe_917526c270ac.slice/crio-7c88fe54d34fa6c4b27ffd306f16e2b2f58e02078697fa809b2443e536f10c4b.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:15:17.430338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:17.430308    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:17.998364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:17.998329    4852 scope.go:122] "RemoveContainer" containerID="689f97f253fb703f0810459cbc75f902d694e3cb1cc9cef9c5d338c7f3d3a47b"
Oct 30 21:15:18.090543 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.090435    4852 generic.go:356] "Generic (PLEG): container finished" podID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerID="73690a404a2f174d744bf9de42a3f1ab9a77dfc35d767e4830be88a6dea62bc2" exitCode=60
Oct 30 21:15:18.090543 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.090511    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" event={"ID":"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6","Type":"ContainerDied","Data":"73690a404a2f174d744bf9de42a3f1ab9a77dfc35d767e4830be88a6dea62bc2"}
Oct 30 21:15:18.090726 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.090579    4852 scope.go:122] "RemoveContainer" containerID="4e94cbca97c440808ca5d9bed44852b57c3db64d5c644cbfe76f6fe89b638267"
Oct 30 21:15:18.091171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.091152    4852 scope.go:122] "RemoveContainer" containerID="73690a404a2f174d744bf9de42a3f1ab9a77dfc35d767e4830be88a6dea62bc2"
Oct 30 21:15:18.091587 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:18.091567    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h_odh-ai-gateway-infra(8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6"
Oct 30 21:15:18.094323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.094300    4852 generic.go:356] "Generic (PLEG): container finished" podID="7188c90b-a928-4434-a988-be7edecbdd9d" containerID="7dd62810f4f4907d5f55d09227f3efe0776244feec39fabb382050ffe6591544" exitCode=60
Oct 30 21:15:18.094451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.094370    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerDied","Data":"7dd62810f4f4907d5f55d09227f3efe0776244feec39fabb382050ffe6591544"}
Oct 30 21:15:18.094796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.094778    4852 scope.go:122] "RemoveContainer" containerID="7dd62810f4f4907d5f55d09227f3efe0776244feec39fabb382050ffe6591544"
Oct 30 21:15:18.095150 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:18.095127    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk_odh-ai-gateway-infra(7188c90b-a928-4434-a988-be7edecbdd9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" podUID="7188c90b-a928-4434-a988-be7edecbdd9d"
Oct 30 21:15:18.096566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.096544    4852 generic.go:356] "Generic (PLEG): container finished" podID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerID="c08aabed2ece661cb59452fb59c98c6ee7d40d58b99e27e89ecb0568d9d86698" exitCode=60
Oct 30 21:15:18.096655 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.096607    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" event={"ID":"cb636f54-64f0-4fe4-86c3-79864ec7a709","Type":"ContainerDied","Data":"c08aabed2ece661cb59452fb59c98c6ee7d40d58b99e27e89ecb0568d9d86698"}
Oct 30 21:15:18.097067 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.097045    4852 scope.go:122] "RemoveContainer" containerID="c08aabed2ece661cb59452fb59c98c6ee7d40d58b99e27e89ecb0568d9d86698"
Oct 30 21:15:18.097512 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:18.097488    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx_odh-ai-gateway-infra(cb636f54-64f0-4fe4-86c3-79864ec7a709)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709"
Oct 30 21:15:18.098661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.098631    4852 generic.go:356] "Generic (PLEG): container finished" podID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerID="54fa9c660b55b795a9244713e5413d67a69b8e4defb09e337828beb6bb94e88a" exitCode=60
Oct 30 21:15:18.098779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.098697    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" event={"ID":"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3","Type":"ContainerDied","Data":"54fa9c660b55b795a9244713e5413d67a69b8e4defb09e337828beb6bb94e88a"}
Oct 30 21:15:18.099116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.099095    4852 scope.go:122] "RemoveContainer" containerID="54fa9c660b55b795a9244713e5413d67a69b8e4defb09e337828beb6bb94e88a"
Oct 30 21:15:18.099449 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:18.099429    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7_odh-ai-gateway-infra(53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3"
Oct 30 21:15:18.100840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.100821    4852 generic.go:356] "Generic (PLEG): container finished" podID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerID="7c88fe54d34fa6c4b27ffd306f16e2b2f58e02078697fa809b2443e536f10c4b" exitCode=60
Oct 30 21:15:18.100933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.100888    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerDied","Data":"7c88fe54d34fa6c4b27ffd306f16e2b2f58e02078697fa809b2443e536f10c4b"}
Oct 30 21:15:18.101213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.101197    4852 scope.go:122] "RemoveContainer" containerID="7c88fe54d34fa6c4b27ffd306f16e2b2f58e02078697fa809b2443e536f10c4b"
Oct 30 21:15:18.101522 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:18.101502    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g_odh-ai-gateway-infra(779bd1e9-c4ac-4d54-bbbe-917526c270ac)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac"
Oct 30 21:15:18.116948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.116918    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"]
Oct 30 21:15:18.137142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.137119    4852 scope.go:122] "RemoveContainer" containerID="b6eebcec73a7ddecc1f8f0d7ac85f3a694df4bdd68f1a7c0188b557a100fc3d6"
Oct 30 21:15:18.141537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.141508    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"]
Oct 30 21:15:18.154468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.154439    4852 scope.go:122] "RemoveContainer" containerID="4fb3057fba08799838c6cc586f9d30dd9de4fe5b135854166b3c8d4a4639bc9f"
Oct 30 21:15:18.212531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.212457    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:18.217263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.217238    4852 scope.go:122] "RemoveContainer" containerID="826b70f11032ec77ff5f8041182b61cc671b843820fedf3a3bca437f23842563"
Oct 30 21:15:18.232895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.232868    4852 scope.go:122] "RemoveContainer" containerID="bf1e53490dea71d5db80ba42f894a1c1fa9bb8340eba9624c60f0830fa63cef7"
Oct 30 21:15:18.334575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.321267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:18.997741 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.997704    4852 scope.go:122] "RemoveContainer" containerID="1595873a3771c311c5de2ec1b6265d7f5da3d4de60b297e66de7de7bcfc93c91"
Oct 30 21:15:18.997908 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.997802    4852 scope.go:122] "RemoveContainer" containerID="49904dcb58543d08e021cf593f869620cbcbdc60a32522e3bafb70ffd8881b35"
Oct 30 21:15:18.998002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:18.997907    4852 scope.go:122] "RemoveContainer" containerID="1994a232f0102145507ed246482376172f0e008b106ff88ab5fdd294d988fb80"
Oct 30 21:15:19.114273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.114244    4852 generic.go:356] "Generic (PLEG): container finished" podID="135c52c6-069b-474f-996f-d463154006b0" containerID="4727f5601fd92600882a9a9335c2024f499a3cdc068949f7e66c56c8e3c10007" exitCode=60
Oct 30 21:15:19.114843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.114311    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" event={"ID":"135c52c6-069b-474f-996f-d463154006b0","Type":"ContainerDied","Data":"4727f5601fd92600882a9a9335c2024f499a3cdc068949f7e66c56c8e3c10007"}
Oct 30 21:15:19.114843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.114375    4852 scope.go:122] "RemoveContainer" containerID="689f97f253fb703f0810459cbc75f902d694e3cb1cc9cef9c5d338c7f3d3a47b"
Oct 30 21:15:19.114961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.114860    4852 scope.go:122] "RemoveContainer" containerID="4727f5601fd92600882a9a9335c2024f499a3cdc068949f7e66c56c8e3c10007"
Oct 30 21:15:19.115451 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:19.115277    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7_odh-ai-gateway-infra(135c52c6-069b-474f-996f-d463154006b0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" podUID="135c52c6-069b-474f-996f-d463154006b0"
Oct 30 21:15:19.132775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.132741    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"]
Oct 30 21:15:19.161950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.161913    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"]
Oct 30 21:15:19.205106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.205071    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"]
Oct 30 21:15:19.205502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.205448    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:19.431211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.431184    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:19.482045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.480180    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/779bd1e9-c4ac-4d54-bbbe-917526c270ac-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/779bd1e9-c4ac-4d54-bbbe-917526c270ac-maas-api-service-ca\") pod \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") "
Oct 30 21:15:19.482045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.480379    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/779bd1e9-c4ac-4d54-bbbe-917526c270ac-kube-api-access-926rz\" (UniqueName: \"kubernetes.io/projected/779bd1e9-c4ac-4d54-bbbe-917526c270ac-kube-api-access-926rz\") pod \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\" (UID: \"779bd1e9-c4ac-4d54-bbbe-917526c270ac\") "
Oct 30 21:15:19.482045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.481864    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/779bd1e9-c4ac-4d54-bbbe-917526c270ac-maas-api-service-ca" pod "779bd1e9-c4ac-4d54-bbbe-917526c270ac" (UID: "779bd1e9-c4ac-4d54-bbbe-917526c270ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:19.552479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.552426    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779bd1e9-c4ac-4d54-bbbe-917526c270ac-kube-api-access-926rz" pod "779bd1e9-c4ac-4d54-bbbe-917526c270ac" (UID: "779bd1e9-c4ac-4d54-bbbe-917526c270ac"). InnerVolumeSpecName "kube-api-access-926rz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:19.584060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.583869    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/779bd1e9-c4ac-4d54-bbbe-917526c270ac-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.584060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.583916    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-926rz\" (UniqueName: \"kubernetes.io/projected/779bd1e9-c4ac-4d54-bbbe-917526c270ac-kube-api-access-926rz\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.602237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.602209    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:19.608665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.608581    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:19.618635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.618575    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:19.626289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.626262    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:19.789207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789173    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7188c90b-a928-4434-a988-be7edecbdd9d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7188c90b-a928-4434-a988-be7edecbdd9d-maas-api-service-ca\") pod \"7188c90b-a928-4434-a988-be7edecbdd9d\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") "
Oct 30 21:15:19.789403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789230    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7188c90b-a928-4434-a988-be7edecbdd9d-kube-api-access-l8jfv\" (UniqueName: \"kubernetes.io/projected/7188c90b-a928-4434-a988-be7edecbdd9d-kube-api-access-l8jfv\") pod \"7188c90b-a928-4434-a988-be7edecbdd9d\" (UID: \"7188c90b-a928-4434-a988-be7edecbdd9d\") "
Oct 30 21:15:19.789403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789287    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-maas-api-service-ca\") pod \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") "
Oct 30 21:15:19.789403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789326    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb636f54-64f0-4fe4-86c3-79864ec7a709-kube-api-access-2qsmv\" (UniqueName: \"kubernetes.io/projected/cb636f54-64f0-4fe4-86c3-79864ec7a709-kube-api-access-2qsmv\") pod \"cb636f54-64f0-4fe4-86c3-79864ec7a709\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") "
Oct 30 21:15:19.789588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789529    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-kube-api-access-9t7x6\" (UniqueName: \"kubernetes.io/projected/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-kube-api-access-9t7x6\") pod \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") "
Oct 30 21:15:19.789588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789557    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7188c90b-a928-4434-a988-be7edecbdd9d-maas-api-service-ca" pod "7188c90b-a928-4434-a988-be7edecbdd9d" (UID: "7188c90b-a928-4434-a988-be7edecbdd9d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:19.789726 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789672    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-maas-api-service-ca" pod "53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" (UID: "53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:19.789726 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789703    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-kube-api-access-hdxfv\" (UniqueName: \"kubernetes.io/projected/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-kube-api-access-hdxfv\") pod \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\" (UID: \"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3\") "
Oct 30 21:15:19.789852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789773    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-maas-api-service-ca\") pod \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\" (UID: \"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6\") "
Oct 30 21:15:19.789852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.789813    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb636f54-64f0-4fe4-86c3-79864ec7a709-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb636f54-64f0-4fe4-86c3-79864ec7a709-maas-api-service-ca\") pod \"cb636f54-64f0-4fe4-86c3-79864ec7a709\" (UID: \"cb636f54-64f0-4fe4-86c3-79864ec7a709\") "
Oct 30 21:15:19.790369 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.790236    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-maas-api-service-ca" pod "8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" (UID: "8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:19.790540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.790504    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb636f54-64f0-4fe4-86c3-79864ec7a709-maas-api-service-ca" pod "cb636f54-64f0-4fe4-86c3-79864ec7a709" (UID: "cb636f54-64f0-4fe4-86c3-79864ec7a709"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:19.790754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.790733    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.790754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.790755    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb636f54-64f0-4fe4-86c3-79864ec7a709-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.790955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.790764    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7188c90b-a928-4434-a988-be7edecbdd9d-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.790955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.790773    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.791847 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.791829    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-kube-api-access-hdxfv" pod "53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" (UID: "53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3"). InnerVolumeSpecName "kube-api-access-hdxfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:19.791932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.791857    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-kube-api-access-9t7x6" pod "8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" (UID: "8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6"). InnerVolumeSpecName "kube-api-access-9t7x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:19.792010 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.791997    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb636f54-64f0-4fe4-86c3-79864ec7a709-kube-api-access-2qsmv" pod "cb636f54-64f0-4fe4-86c3-79864ec7a709" (UID: "cb636f54-64f0-4fe4-86c3-79864ec7a709"). InnerVolumeSpecName "kube-api-access-2qsmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:19.792047 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.792002    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7188c90b-a928-4434-a988-be7edecbdd9d-kube-api-access-l8jfv" pod "7188c90b-a928-4434-a988-be7edecbdd9d" (UID: "7188c90b-a928-4434-a988-be7edecbdd9d"). InnerVolumeSpecName "kube-api-access-l8jfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:19.892429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.892334    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t7x6\" (UniqueName: \"kubernetes.io/projected/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6-kube-api-access-9t7x6\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.892429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.892364    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdxfv\" (UniqueName: \"kubernetes.io/projected/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3-kube-api-access-hdxfv\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.892429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.892374    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8jfv\" (UniqueName: \"kubernetes.io/projected/7188c90b-a928-4434-a988-be7edecbdd9d-kube-api-access-l8jfv\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:19.892429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:19.892394    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qsmv\" (UniqueName: \"kubernetes.io/projected/cb636f54-64f0-4fe4-86c3-79864ec7a709-kube-api-access-2qsmv\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:20.033631 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.033592    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:20.135340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.135308    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"
Oct 30 21:15:20.135855 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.135334    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7" event={"ID":"53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3","Type":"ContainerDied","Data":"59367ae0aaead65f36563db8335da0b64228e5a89e74dbff82ec97015a6fe37b"}
Oct 30 21:15:20.135855 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.135456    4852 scope.go:122] "RemoveContainer" containerID="54fa9c660b55b795a9244713e5413d67a69b8e4defb09e337828beb6bb94e88a"
Oct 30 21:15:20.137284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.137258    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"
Oct 30 21:15:20.137473 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.137285    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g" event={"ID":"779bd1e9-c4ac-4d54-bbbe-917526c270ac","Type":"ContainerDied","Data":"d87c30852cd8d0f6958dda3361e891875562a2443dd4729eabd4f8e348ce7504"}
Oct 30 21:15:20.140211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.140124    4852 generic.go:356] "Generic (PLEG): container finished" podID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerID="58b61f916f6c105993701e49045aabeae1a9bf6b8665c27f62cc58a1f0d91c66" exitCode=60
Oct 30 21:15:20.140211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.140180    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" event={"ID":"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee","Type":"ContainerDied","Data":"58b61f916f6c105993701e49045aabeae1a9bf6b8665c27f62cc58a1f0d91c66"}
Oct 30 21:15:20.140695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.140678    4852 scope.go:122] "RemoveContainer" containerID="58b61f916f6c105993701e49045aabeae1a9bf6b8665c27f62cc58a1f0d91c66"
Oct 30 21:15:20.141035 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:20.141005    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg_odh-ai-gateway-infra(94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee"
Oct 30 21:15:20.146118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.146036    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerStarted","Data":"3098067d66ac2fea1603869437412de57cf370d43dce83e84ead2927980c5b5c"}
Oct 30 21:15:20.152105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.151995    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h" event={"ID":"8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6","Type":"ContainerDied","Data":"a517540ae44f77b8f14a6da2207cfc2fa744147aa769b6b1a2e4325867f155db"}
Oct 30 21:15:20.152889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.152482    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"
Oct 30 21:15:20.154617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.154580    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"
Oct 30 21:15:20.154734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.154619    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk" event={"ID":"7188c90b-a928-4434-a988-be7edecbdd9d","Type":"ContainerDied","Data":"cb0dfc7dc7736180f90289b9e7c78a4baf2eaf7a162c09cb1aeb41787d960e17"}
Oct 30 21:15:20.156642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.156625    4852 scope.go:122] "RemoveContainer" containerID="7c88fe54d34fa6c4b27ffd306f16e2b2f58e02078697fa809b2443e536f10c4b"
Oct 30 21:15:20.156856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.156836    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx" event={"ID":"cb636f54-64f0-4fe4-86c3-79864ec7a709","Type":"ContainerDied","Data":"4e77879fd8afa479e08c92bbe500930f4c2bf008d8adcd9ebfd7ef4e36dec48f"}
Oct 30 21:15:20.156955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.156928    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"
Oct 30 21:15:20.160217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.160194    4852 generic.go:356] "Generic (PLEG): container finished" podID="f13a424a-d872-4016-82de-9d49f62e9998" containerID="9642edc63629765f38e2f6e682d3bcfaf6349fd5f3232864cc6ce5abc99546ed" exitCode=60
Oct 30 21:15:20.160304 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.160237    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" event={"ID":"f13a424a-d872-4016-82de-9d49f62e9998","Type":"ContainerDied","Data":"9642edc63629765f38e2f6e682d3bcfaf6349fd5f3232864cc6ce5abc99546ed"}
Oct 30 21:15:20.160716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.160700    4852 scope.go:122] "RemoveContainer" containerID="9642edc63629765f38e2f6e682d3bcfaf6349fd5f3232864cc6ce5abc99546ed"
Oct 30 21:15:20.161009 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:20.160978    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl_odh-ai-gateway-infra(f13a424a-d872-4016-82de-9d49f62e9998)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" podUID="f13a424a-d872-4016-82de-9d49f62e9998"
Oct 30 21:15:20.167530 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.167506    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"]
Oct 30 21:15:20.181367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.181336    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-8e4acb-29889915-p7s6g"]
Oct 30 21:15:20.204478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.203935    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"]
Oct 30 21:15:20.212214 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.212182    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-53aa47-29889915-kzkm7"]
Oct 30 21:15:20.230006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.229973    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"]
Oct 30 21:15:20.274252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.274216    4852 scope.go:122] "RemoveContainer" containerID="49904dcb58543d08e021cf593f869620cbcbdc60a32522e3bafb70ffd8881b35"
Oct 30 21:15:20.294253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.294223    4852 scope.go:122] "RemoveContainer" containerID="73690a404a2f174d744bf9de42a3f1ab9a77dfc35d767e4830be88a6dea62bc2"
Oct 30 21:15:20.306524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.306497    4852 scope.go:122] "RemoveContainer" containerID="7dd62810f4f4907d5f55d09227f3efe0776244feec39fabb382050ffe6591544"
Oct 30 21:15:20.321146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.321127    4852 scope.go:122] "RemoveContainer" containerID="c08aabed2ece661cb59452fb59c98c6ee7d40d58b99e27e89ecb0568d9d86698"
Oct 30 21:15:20.351737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.350955    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"]
Oct 30 21:15:20.353990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.353958    4852 scope.go:122] "RemoveContainer" containerID="1994a232f0102145507ed246482376172f0e008b106ff88ab5fdd294d988fb80"
Oct 30 21:15:20.375982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.375770    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-bc102e-29889915-5r4gx"]
Oct 30 21:15:20.382007 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.381980    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"]
Oct 30 21:15:20.390342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.389594    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889915-zmvzk"]
Oct 30 21:15:20.398685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.398605    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"]
Oct 30 21:15:20.406337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.406313    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889915-g8w2h"]
Oct 30 21:15:20.440430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.440378    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:20.508884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.508848    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/135c52c6-069b-474f-996f-d463154006b0-kube-api-access-vktrd\" (UniqueName: \"kubernetes.io/projected/135c52c6-069b-474f-996f-d463154006b0-kube-api-access-vktrd\") pod \"135c52c6-069b-474f-996f-d463154006b0\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") "
Oct 30 21:15:20.509082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.509035    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/135c52c6-069b-474f-996f-d463154006b0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/135c52c6-069b-474f-996f-d463154006b0-maas-api-service-ca\") pod \"135c52c6-069b-474f-996f-d463154006b0\" (UID: \"135c52c6-069b-474f-996f-d463154006b0\") "
Oct 30 21:15:20.509526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.509492    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/135c52c6-069b-474f-996f-d463154006b0-maas-api-service-ca" pod "135c52c6-069b-474f-996f-d463154006b0" (UID: "135c52c6-069b-474f-996f-d463154006b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:20.510058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.510032    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/135c52c6-069b-474f-996f-d463154006b0-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:20.511748 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.511715    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/135c52c6-069b-474f-996f-d463154006b0-kube-api-access-vktrd" pod "135c52c6-069b-474f-996f-d463154006b0" (UID: "135c52c6-069b-474f-996f-d463154006b0"). InnerVolumeSpecName "kube-api-access-vktrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:20.611289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.611252    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vktrd\" (UniqueName: \"kubernetes.io/projected/135c52c6-069b-474f-996f-d463154006b0-kube-api-access-vktrd\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:20.970284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:20.970249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:21.169873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.169836    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"
Oct 30 21:15:21.169873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.169858    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7" event={"ID":"135c52c6-069b-474f-996f-d463154006b0","Type":"ContainerDied","Data":"368610f0f5378ef7b5f208f1a4c3f2183dc9b72d89a603cb68a6f023433a4820"}
Oct 30 21:15:21.170357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.169920    4852 scope.go:122] "RemoveContainer" containerID="4727f5601fd92600882a9a9335c2024f499a3cdc068949f7e66c56c8e3c10007"
Oct 30 21:15:21.194201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.194173    4852 generic.go:356] "Generic (PLEG): container finished" podID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerID="3098067d66ac2fea1603869437412de57cf370d43dce83e84ead2927980c5b5c" exitCode=60
Oct 30 21:15:21.194321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.194226    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerDied","Data":"3098067d66ac2fea1603869437412de57cf370d43dce83e84ead2927980c5b5c"}
Oct 30 21:15:21.194804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.194788    4852 scope.go:122] "RemoveContainer" containerID="3098067d66ac2fea1603869437412de57cf370d43dce83e84ead2927980c5b5c"
Oct 30 21:15:21.195074 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:21.195057    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk_odh-ai-gateway-infra(8d888772-04e0-4d87-8b74-fe08bcfd2a9d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d"
Oct 30 21:15:21.197485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.197468    4852 scope.go:122] "RemoveContainer" containerID="1595873a3771c311c5de2ec1b6265d7f5da3d4de60b297e66de7de7bcfc93c91"
Oct 30 21:15:21.204654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.204633    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"]
Oct 30 21:15:21.210208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.210176    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-99668e-29889915-4cnb7"]
Oct 30 21:15:21.245766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.245691    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"]
Oct 30 21:15:21.280504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.280464    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"]
Oct 30 21:15:21.308413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.308364    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"]
Oct 30 21:15:21.377458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.375599    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"]
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379301    4852 cpu_manager.go:414] "containerMap: removing container" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379325    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379346    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379352    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379359    4852 cpu_manager.go:414] "containerMap: removing container" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379364    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379402    4852 cpu_manager.go:414] "containerMap: removing container" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379410    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379431    4852 cpu_manager.go:414] "containerMap: removing container" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379437    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:21.379442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379448    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379455    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379471    4852 cpu_manager.go:414] "containerMap: removing container" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379478    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379486    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379492    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379513    4852 cpu_manager.go:414] "containerMap: removing container" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379519    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379530    4852 cpu_manager.go:414] "containerMap: removing container" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379536    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379551    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379558    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379574    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379580    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379601    4852 cpu_manager.go:414] "containerMap: removing container" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379609    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379813    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379832    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379843    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379856    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379866    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379876    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379884    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379894    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379905    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379914    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379932    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379941    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:21.380201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.379948    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:21.390058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.390028    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.426729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.426693    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"]
Oct 30 21:15:21.439079 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439044    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439105    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439344    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439398    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db966d6e-90cc-450c-9842-18807aac3c9d-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439474    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439507    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq22s\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-kube-api-access-sq22s\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439539    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db966d6e-90cc-450c-9842-18807aac3c9d-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439601    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.439645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.439632    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.488078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.488040    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:21.554526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.554485    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-kube-api-access-p95lp\" (UniqueName: \"kubernetes.io/projected/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-kube-api-access-p95lp\") pod \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") "
Oct 30 21:15:21.554718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.554595    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-maas-api-service-ca\") pod \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\" (UID: \"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee\") "
Oct 30 21:15:21.555311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555282    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555342    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555539    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555574    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db966d6e-90cc-450c-9842-18807aac3c9d-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555646    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555678    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq22s\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-kube-api-access-sq22s\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555711    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db966d6e-90cc-450c-9842-18807aac3c9d-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555775    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.555916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.555804    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.556324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.556301    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.558471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.557023    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-maas-api-service-ca" pod "94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" (UID: "94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:21.558471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.557302    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.558471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.557870    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.558914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.558888    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db966d6e-90cc-450c-9842-18807aac3c9d-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.559933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.559910    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db966d6e-90cc-450c-9842-18807aac3c9d-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.560175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.560154    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.561344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.561319    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.561888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.561864    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-kube-api-access-p95lp" pod "94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" (UID: "94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee"). InnerVolumeSpecName "kube-api-access-p95lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:21.572156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.571973    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq22s\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-kube-api-access-sq22s\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.573893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.573864    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.576424 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:21.576379    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:15:21.577914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.577893    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:21.660403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.656547    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f13a424a-d872-4016-82de-9d49f62e9998-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13a424a-d872-4016-82de-9d49f62e9998-maas-api-service-ca\") pod \"f13a424a-d872-4016-82de-9d49f62e9998\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") "
Oct 30 21:15:21.660403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.656971    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f13a424a-d872-4016-82de-9d49f62e9998-kube-api-access-hpj74\" (UniqueName: \"kubernetes.io/projected/f13a424a-d872-4016-82de-9d49f62e9998-kube-api-access-hpj74\") pod \"f13a424a-d872-4016-82de-9d49f62e9998\" (UID: \"f13a424a-d872-4016-82de-9d49f62e9998\") "
Oct 30 21:15:21.660403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.657029    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13a424a-d872-4016-82de-9d49f62e9998-maas-api-service-ca" pod "f13a424a-d872-4016-82de-9d49f62e9998" (UID: "f13a424a-d872-4016-82de-9d49f62e9998"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:21.660403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.657668    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p95lp\" (UniqueName: \"kubernetes.io/projected/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-kube-api-access-p95lp\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:21.660403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.657690    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:21.660403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.657707    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13a424a-d872-4016-82de-9d49f62e9998-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:21.660403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.659728    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13a424a-d872-4016-82de-9d49f62e9998-kube-api-access-hpj74" pod "f13a424a-d872-4016-82de-9d49f62e9998" (UID: "f13a424a-d872-4016-82de-9d49f62e9998"). InnerVolumeSpecName "kube-api-access-hpj74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:21.720492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.720459    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:21.758915 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:21.758876    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpj74\" (UniqueName: \"kubernetes.io/projected/f13a424a-d872-4016-82de-9d49f62e9998-kube-api-access-hpj74\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:22.011118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.011084    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="135c52c6-069b-474f-996f-d463154006b0" path="/var/lib/kubelet/pods/135c52c6-069b-474f-996f-d463154006b0/volumes"
Oct 30 21:15:22.011656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.011638    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" path="/var/lib/kubelet/pods/53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3/volumes"
Oct 30 21:15:22.012100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.012085    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" path="/var/lib/kubelet/pods/7188c90b-a928-4434-a988-be7edecbdd9d/volumes"
Oct 30 21:15:22.013094 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.013068    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" path="/var/lib/kubelet/pods/779bd1e9-c4ac-4d54-bbbe-917526c270ac/volumes"
Oct 30 21:15:22.013859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.013838    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" path="/var/lib/kubelet/pods/8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6/volumes"
Oct 30 21:15:22.014286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.014272    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" path="/var/lib/kubelet/pods/cb636f54-64f0-4fe4-86c3-79864ec7a709/volumes"
Oct 30 21:15:22.055039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.055008    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"]
Oct 30 21:15:22.059165 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:22.059130    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb966d6e_90cc_450c_9842_18807aac3c9d.slice/crio-6ebc1df0ca861ffd52c28787821bd39a838214b01f0e908336a5db20adc94012 WatchSource:0}: Error finding container 6ebc1df0ca861ffd52c28787821bd39a838214b01f0e908336a5db20adc94012: Status 404 returned error can't find the container with id 6ebc1df0ca861ffd52c28787821bd39a838214b01f0e908336a5db20adc94012
Oct 30 21:15:22.063097 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.063057    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:22.063210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.063146    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:22.063278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.063221    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:22.203585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.203549    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:22.207644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.207618    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"
Oct 30 21:15:22.207644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.207632    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl" event={"ID":"f13a424a-d872-4016-82de-9d49f62e9998","Type":"ContainerDied","Data":"12a310a17ee2c0e140f13a8e9544ae473e495bac1433858519ce4e1094d4f243"}
Oct 30 21:15:22.207847 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.207758    4852 scope.go:122] "RemoveContainer" containerID="9642edc63629765f38e2f6e682d3bcfaf6349fd5f3232864cc6ce5abc99546ed"
Oct 30 21:15:22.209746 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.209704    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" event={"ID":"db966d6e-90cc-450c-9842-18807aac3c9d","Type":"ContainerStarted","Data":"6ebc1df0ca861ffd52c28787821bd39a838214b01f0e908336a5db20adc94012"}
Oct 30 21:15:22.211872 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.211850    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg" event={"ID":"94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee","Type":"ContainerDied","Data":"dc077352d2e261f25ca889ceb51264ffc5159a7ac0d592934795048c3a474220"}
Oct 30 21:15:22.211992 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.211930    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"
Oct 30 21:15:22.233562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.233397    4852 scope.go:122] "RemoveContainer" containerID="58b61f916f6c105993701e49045aabeae1a9bf6b8665c27f62cc58a1f0d91c66"
Oct 30 21:15:22.267729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.267653    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podStartSLOduration=1.26763335 podStartE2EDuration="1.26763335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:22.250802892 +0000 UTC m=+2230.365537086" watchObservedRunningTime="2026-10-30 21:15:22.26763335 +0000 UTC m=+2230.382367541"
Oct 30 21:15:22.273951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.273918    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"]
Oct 30 21:15:22.277796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.277768    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-00909f-29889915-jl7jg"]
Oct 30 21:15:22.297786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.297759    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"]
Oct 30 21:15:22.306745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.306718    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-d95261-29889915-rbpgl"]
Oct 30 21:15:22.429213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.429183    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:22.580280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.580192    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-kube-api-access-xjck9\" (UniqueName: \"kubernetes.io/projected/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-kube-api-access-xjck9\") pod \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") "
Oct 30 21:15:22.580280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.580252    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-maas-api-service-ca\") pod \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\" (UID: \"8d888772-04e0-4d87-8b74-fe08bcfd2a9d\") "
Oct 30 21:15:22.580649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.580624    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-maas-api-service-ca" pod "8d888772-04e0-4d87-8b74-fe08bcfd2a9d" (UID: "8d888772-04e0-4d87-8b74-fe08bcfd2a9d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:15:22.581177 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.581156    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:22.582674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.582635    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-kube-api-access-xjck9" pod "8d888772-04e0-4d87-8b74-fe08bcfd2a9d" (UID: "8d888772-04e0-4d87-8b74-fe08bcfd2a9d"). InnerVolumeSpecName "kube-api-access-xjck9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:22.682449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.682380    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjck9\" (UniqueName: \"kubernetes.io/projected/8d888772-04e0-4d87-8b74-fe08bcfd2a9d-kube-api-access-xjck9\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:22.721260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:22.721220    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:23.013362 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.013330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:23.239954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.239913    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk" event={"ID":"8d888772-04e0-4d87-8b74-fe08bcfd2a9d","Type":"ContainerDied","Data":"9ab785cf599ee82d9e5cea238eba8a1d295502336044cf63ffa0f12b3368feeb"}
Oct 30 21:15:23.240398 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.240012    4852 scope.go:122] "RemoveContainer" containerID="3098067d66ac2fea1603869437412de57cf370d43dce83e84ead2927980c5b5c"
Oct 30 21:15:23.240398 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.239933    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"
Oct 30 21:15:23.244664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.244627    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" event={"ID":"db966d6e-90cc-450c-9842-18807aac3c9d","Type":"ContainerStarted","Data":"c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770"}
Oct 30 21:15:23.299379 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.299340    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"]
Oct 30 21:15:23.308077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.308047    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889915-7xxzk"]
Oct 30 21:15:23.722832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.722739    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 21:15:23.722832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:23.722803    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 21:15:24.006948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.006917    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" path="/var/lib/kubelet/pods/8d888772-04e0-4d87-8b74-fe08bcfd2a9d/volumes"
Oct 30 21:15:24.007452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.007437    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" path="/var/lib/kubelet/pods/94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee/volumes"
Oct 30 21:15:24.007881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.007868    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f13a424a-d872-4016-82de-9d49f62e9998" path="/var/lib/kubelet/pods/f13a424a-d872-4016-82de-9d49f62e9998/volumes"
Oct 30 21:15:24.128360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.128329    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:24.224838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.222931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:24.626588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.626554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:24.726411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.721956    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 21:15:24.726411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:24.722021    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 21:15:25.640531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:25.640497    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:25.720929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:25.720896    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 21:15:25.721138 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:25.720947    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 21:15:26.017688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:26.017658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:26.721851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:26.721811    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 21:15:26.722273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:26.721866    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 21:15:27.079669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:27.079629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:27.721538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:27.721494    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 30 21:15:27.721724 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:27.721554    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 30 21:15:28.293420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:28.293357    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:28.726252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:28.726172    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:28.726614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:28.726584    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:28.727234 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:28.727211    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:15:29.432881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:29.432841    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:30.320209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:30.320175    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:31.193740 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:31.193700    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:32.151112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:32.151042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:32.943751 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:32.943718    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:33.088397 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:33.088332    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:33.207488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:33.207360    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:33.739685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:33.739654    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:34.656585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:34.656552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:35.550729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:35.550696    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:36.395678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.395641    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"]
Oct 30 21:15:36.397468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397443    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:15:36.397468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397468    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397489    4852 cpu_manager.go:414] "containerMap: removing container" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397497    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53c5bd5e-97a9-4533-b149-2ce8c9a3c4f3" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397552    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397560    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397572    4852 cpu_manager.go:414] "containerMap: removing container" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397578    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397601    4852 cpu_manager.go:414] "containerMap: removing container" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397608    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397627    4852 cpu_manager.go:414] "containerMap: removing container" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397633    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397649    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397656    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397666    4852 cpu_manager.go:414] "containerMap: removing container" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397673    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397684    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397692    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397709    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:15:36.397707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397715    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397728    4852 cpu_manager.go:414] "containerMap: removing container" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397734    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397752    4852 cpu_manager.go:414] "containerMap: removing container" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397759    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397941    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397955    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397964    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="7188c90b-a928-4434-a988-be7edecbdd9d" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397976    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c8ca4b3-07e9-485d-bd38-c74d1e7ad1a6" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.397994    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb636f54-64f0-4fe4-86c3-79864ec7a709" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.398007    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.398016    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.398026    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.398036    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.398046    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.398055    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="779bd1e9-c4ac-4d54-bbbe-917526c270ac" containerName="cleanup"
Oct 30 21:15:36.398912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.398064    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="135c52c6-069b-474f-996f-d463154006b0" containerName="cleanup"
Oct 30 21:15:36.404049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.404025    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.422449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.422415    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:36.423324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.422638    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"]
Oct 30 21:15:36.501333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.501285    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.501537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.501456    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.501707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.501682    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.501757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.501724    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.501872 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.501855    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.501957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.501940    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.501994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.501986    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.502039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.502023    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0975483-3126-4e3a-8cf5-9eba54e64198-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.502076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.502053    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpln8\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-kube-api-access-fpln8\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603133    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0975483-3126-4e3a-8cf5-9eba54e64198-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603176    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpln8\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-kube-api-access-fpln8\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603233    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603274    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603424    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603450    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603670 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603585    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603739 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603693    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603739    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603855 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603811    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-data\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603874    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-credential-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.603991 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.603969    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-certs\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.604033 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.604020    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0975483-3126-4e3a-8cf5-9eba54e64198-istiod-ca-cert\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.604169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.604147    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-socket\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.605811 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.605783    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-envoy\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.606216 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.606195    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-podinfo\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.613097 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.612826    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-token\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.613097 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.613053    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpln8\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-kube-api-access-fpln8\") pod \"e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") " pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:36.733381 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:36.733280    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:37.035765 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.035733    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"]
Oct 30 21:15:37.036161 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:15:37.036138    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0975483_3126_4e3a_8cf5_9eba54e64198.slice/crio-e479096af085059bbcc7aab57ee5abcf417834c53d8e662185df8bb819f05773 WatchSource:0}: Error finding container e479096af085059bbcc7aab57ee5abcf417834c53d8e662185df8bb819f05773: Status 404 returned error can't find the container with id e479096af085059bbcc7aab57ee5abcf417834c53d8e662185df8bb819f05773
Oct 30 21:15:37.039856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.039815    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:37.039957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.039923    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:37.039996 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.039965    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:15:37.335747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.335605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:37.439697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.439658    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" event={"ID":"d0975483-3126-4e3a-8cf5-9eba54e64198","Type":"ContainerStarted","Data":"76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b"}
Oct 30 21:15:37.442212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.439727    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" event={"ID":"d0975483-3126-4e3a-8cf5-9eba54e64198","Type":"ContainerStarted","Data":"e479096af085059bbcc7aab57ee5abcf417834c53d8e662185df8bb819f05773"}
Oct 30 21:15:37.467609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.467552    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" podStartSLOduration=1.467535887 podStartE2EDuration="1.467535887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:37.466765941 +0000 UTC m=+2245.581500133" watchObservedRunningTime="2026-10-30 21:15:37.467535887 +0000 UTC m=+2245.582270077"
Oct 30 21:15:37.733780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:37.733699    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:38.098582 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:38.098543    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:38.735166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:38.735131    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 21:15:38.735620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:38.735184    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 21:15:39.198167 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:39.198133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:39.690685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:39.690178    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:39.734350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:39.734315    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 21:15:39.734570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:39.734374    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 21:15:39.798378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:39.798342    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:40.416511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:40.415758    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:40.734637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:40.734553    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 21:15:40.734637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:40.734609    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 21:15:41.241058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:41.241021    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:41.733685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:41.733645    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 21:15:41.733891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:41.733714    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 21:15:41.874064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:41.874026    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:42.224055 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:42.224022    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:42.734129 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:42.734095    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused" start-of-body=
Oct 30 21:15:42.734595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:42.734150    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.172:15021/healthz/ready\": dial tcp 10.133.0.172:15021: connect: connection refused"
Oct 30 21:15:43.104237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:43.104201    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:43.215013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:43.214980    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:43.813953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:43.813927    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:43.814487 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:43.814443    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:43.815751 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:43.815720    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:15:43.954848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:43.954817    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:44.694799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:44.694760    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:45.598536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:45.598504    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:46.447498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:46.447427    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:47.187161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:47.187126    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:47.945549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:47.945519    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:48.216511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:48.216311    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:48.319702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:48.319654    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:48.775835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:48.775798    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:49.750726 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:49.748993    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:50.639668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:50.639636    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:51.443929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:51.443897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:51.567062 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:15:51.567023    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:15:52.229142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:52.229106    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:53.135639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:53.135552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:53.505505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:53.505431    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:53.666841 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:53.643240    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:54.327694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:54.327657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:54.599045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:54.598954    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:54.700477 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:54.700441    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:54.798262 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:54.798229    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:55.164280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:55.164245    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:55.621071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:55.621031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:56.502261 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:56.502186    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:57.256889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:57.256850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:58.036402 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:58.036358    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:58.785676 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:58.785640    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:59.636059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:15:59.636019    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:00.573796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:00.573743    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:01.731270 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:01.731230    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:02.864573 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:02.864539    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:03.080277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:03.080202    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:03.195640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:03.195563    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:04.197773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:04.197738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:04.868439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:04.868375    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:05.189299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:05.189214    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:06.025714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:06.025682    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:06.844507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:06.844405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:07.717649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:07.717610    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:08.549017 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:08.548961    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:09.544330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:09.544284    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:10.299211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:10.299176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:10.459876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:10.459832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:11.160810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:11.160777    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:11.300447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:11.300407    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:11.433879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:11.433137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:11.856607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:11.856574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:11.965362 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:11.965327    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.062665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:12.062619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.773775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:12.773673    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:13.824330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:13.824296    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:14.780170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:14.780137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:15.664192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:15.664148    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:16.063442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:16.055276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:16.163801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:16.163760    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:16.621366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:16.621327    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:16.928330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:16.928248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:17.201024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:17.200931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:17.346408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:17.344889    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:17.501031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:17.500943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:18.360735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:18.360677    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:19.536604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:19.536569    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:20.289208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:20.289173    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:21.038876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:21.038848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:21.559813 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:16:21.559773    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:16:21.799218 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:21.799173    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:22.161464 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:22.161431    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:22.341414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:22.332155    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:23.276192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:23.276142    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:23.812887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:23.812847    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:23.914899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:23.914867    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:24.012126 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:24.012089    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:24.184891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:24.184778    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:24.792213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:24.792171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:24.911500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:24.911443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:25.345868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:25.345822    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:25.436884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:25.436805    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:25.520784 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:25.520746    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:25.771553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:25.771504    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:25.882410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:25.879013    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:26.147493 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:26.147405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:27.007302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:27.007266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:27.941404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:27.941350    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:28.773450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:28.773369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:29.515583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:29.515540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:30.238841 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:30.238802    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:30.769141 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:30.769103    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:30.987211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:30.987172    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:31.156684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:31.156599    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:31.476254 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:31.476169    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:31.576679 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:31.576646    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:32.593826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:32.593780    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:32.788292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:32.788232    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:32.899322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:32.899221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:33.491985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:33.491955    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:33.741629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:33.741590    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:33.841950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:33.841914    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:34.826680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:34.826637    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:34.926958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:34.926922    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:35.030410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:35.030079    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:35.415849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:35.415814    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:35.515339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:35.515303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:36.336863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:36.336829    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:37.195688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:37.195651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:37.552287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:37.552251    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:37.650678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:37.650622    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:37.749951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:37.749914    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:37.851084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:37.850994    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:38.263424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:38.262132    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:38.937162 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:38.936997    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:39.028503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:39.028470    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:39.375526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:39.375495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:39.473291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:39.473259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:39.589715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:39.589681    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:40.350645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:40.350594    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:41.708359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:41.708320    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:42.650867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:42.650815    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:43.457877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:43.457843    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:43.577059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:43.577000    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:43.784264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:43.784211    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:43.955920 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:43.953975    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:44.898498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:44.898465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:45.945215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:45.945183    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:46.925430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:46.925364    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:47.029370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:47.029298    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:47.129072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:47.129038    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:47.950371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:47.950338    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:48.750096 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:48.750058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:49.616675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:49.616636    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:50.385845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:50.385808    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:50.683155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:50.683056    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:50.791331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:50.791290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:50.893747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:50.893715    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.116893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:51.116861    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.245563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:51.245505    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.384032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:51.383943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.570121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:51.570083    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:51.686660 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:16:51.686574    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:16:52.166978 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:52.166938    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:52.278585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:52.278530    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:52.781489 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:52.781442    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:52.878772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:52.878740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:54.111063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.111029    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:54.420353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.420213    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:54.532798 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.532760    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:54.659569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.659494    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:54.884472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.884435    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:54.981211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.981166    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"]
Oct 30 21:16:54.982455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.982424    4852 cpu_manager.go:414] "containerMap: removing container" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:16:54.982455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.982453    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d888772-04e0-4d87-8b74-fe08bcfd2a9d" containerName="cleanup"
Oct 30 21:16:54.982730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.982496    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:16:54.982730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.982503    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:16:54.982730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.982694    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="94f3a3e8-2522-4ec2-b6b9-b21d4c4c95ee" containerName="cleanup"
Oct 30 21:16:54.982730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:54.982712    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f13a424a-d872-4016-82de-9d49f62e9998" containerName="cleanup"
Oct 30 21:16:55.000906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.000868    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"]
Oct 30 21:16:55.001452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.001429    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.040651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.040612    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbqs6\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-kube-api-access-mbqs6\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.040848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.040661    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.040904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.040886    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.040950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.040923    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.041005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.040990    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.041047 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.041039    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.041088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.041078    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.041124 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.041112    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.041163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.041149    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.142549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.142451    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.142611    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.142848    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbqs6\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-kube-api-access-mbqs6\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.142874    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.142848    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.142972    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143014    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143078    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143149    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143183    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143208    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143291    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143429    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.143742 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.143602    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.145241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.145223    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.145616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.145601    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.155609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.155577    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.155802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.155779    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbqs6\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-kube-api-access-mbqs6\") pod \"maas-default-gateway-openshift-default-77f556d647-x6r22\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.187345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.187289    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:55.292997 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.292963    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:55.319493 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.319456    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:55.667733 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.667685    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"]
Oct 30 21:16:55.682645 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:16:55.682607    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d3a5f7_4ba6_441f_a79c_cc326998db8e.slice/crio-15f750bd145b5d4c62fdf4441efaf4fbc258afdd5ee2c87d7342c083deda4c67 WatchSource:0}: Error finding container 15f750bd145b5d4c62fdf4441efaf4fbc258afdd5ee2c87d7342c083deda4c67: Status 404 returned error can't find the container with id 15f750bd145b5d4c62fdf4441efaf4fbc258afdd5ee2c87d7342c083deda4c67
Oct 30 21:16:55.685632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.685591    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:16:55.685765 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.685682    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:16:55.685765 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.685729    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:16:55.735005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.734968    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:55.847461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:55.847353    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:56.524403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:56.524356    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:56.739818 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:56.739776    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" event={"ID":"90d3a5f7-4ba6-441f-a79c-cc326998db8e","Type":"ContainerStarted","Data":"b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107"}
Oct 30 21:16:56.740085 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:56.740070    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" event={"ID":"90d3a5f7-4ba6-441f-a79c-cc326998db8e","Type":"ContainerStarted","Data":"15f750bd145b5d4c62fdf4441efaf4fbc258afdd5ee2c87d7342c083deda4c67"}
Oct 30 21:16:56.832253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:56.832125    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podStartSLOduration=2.832101221 podStartE2EDuration="2.832101221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:56.804271429 +0000 UTC m=+2324.919005619" watchObservedRunningTime="2026-10-30 21:16:56.832101221 +0000 UTC m=+2324.946835412"
Oct 30 21:16:57.049970 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:57.049904    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:57.319976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:57.319935    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:16:57.321741 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:57.321716    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-x6r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:16:57.321819 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:57.321769    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:16:58.040326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:58.040287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:58.320596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:58.320508    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-x6r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:16:58.320596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:58.320564    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:16:58.992349 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:58.992319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:59.320123 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:59.320081    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-x6r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:16:59.320599 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:59.320148    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:16:59.850587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:16:59.850552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:00.319994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:00.319961    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-x6r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:17:00.320187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:00.320035    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:17:00.605667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:00.605572    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:01.320172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:01.320133    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-x6r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:17:01.320666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:01.320199    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:17:01.321006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:01.320982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:02.248729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:02.248693    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:02.327537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:02.327494    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-x6r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:17:02.328268 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:02.328219    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:17:03.110586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:03.110548    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:03.320804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:03.320769    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-x6r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused" start-of-body=
Oct 30 21:17:03.321022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:03.320823    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.173:15021/healthz/ready\": dial tcp 10.133.0.173:15021: connect: connection refused"
Oct 30 21:17:03.920795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:03.920759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:04.854976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:04.854918    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:04.964303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:04.964266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:05.141117 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:05.141037    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:17:05.141588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:05.141559    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:17:05.142350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:05.142327    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:17:05.648643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:05.648606    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:06.432792 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:06.432754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:07.142100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:07.142062    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:07.842127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:07.842084    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:08.722853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:08.722808    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:09.520463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:09.520380    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:09.949803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:09.949720    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"]
Oct 30 21:17:09.954629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:09.954599    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:09.970980 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:09.970932    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"]
Oct 30 21:17:10.130681 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.130627    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.130883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.130690    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.130883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.130848    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjqbf\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-kube-api-access-fjqbf\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.130883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.130880    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.131024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.130953    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.131083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.131033    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.131166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.131142    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.131303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.131277    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.131495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.131377    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233230    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233290    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233357    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233415    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233483    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233505    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233656    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjqbf\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-kube-api-access-fjqbf\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233702    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233746    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.233901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233784    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-data\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.234210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.233968    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.234210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.234142    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-credential-socket\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.234302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.234216    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-certs\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.234427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.234365    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.236101 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.236053    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.236690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.236669    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.243340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.243314    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-token\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.244118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.244097    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjqbf\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-kube-api-access-fjqbf\") pod \"maas-default-gateway-openshift-default-77f556d647-xr9v5\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.278532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.278494    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:10.378328 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.378291    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:10.592250 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:17:10.592206    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aee0c1b_24db_4a6e_bbe6_8c3bdb151b4e.slice/crio-4b038dbc343c2b0e1b2b03d5aa5a9b700009a7d84daafe62051b9bc427ab785b WatchSource:0}: Error finding container 4b038dbc343c2b0e1b2b03d5aa5a9b700009a7d84daafe62051b9bc427ab785b: Status 404 returned error can't find the container with id 4b038dbc343c2b0e1b2b03d5aa5a9b700009a7d84daafe62051b9bc427ab785b
Oct 30 21:17:10.592941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.592898    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"]
Oct 30 21:17:10.595477 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.595434    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:17:10.595570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.595520    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:17:10.595570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.595560    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:17:10.906514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.906414    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" event={"ID":"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e","Type":"ContainerStarted","Data":"ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca"}
Oct 30 21:17:10.906514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.906486    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" event={"ID":"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e","Type":"ContainerStarted","Data":"4b038dbc343c2b0e1b2b03d5aa5a9b700009a7d84daafe62051b9bc427ab785b"}
Oct 30 21:17:10.966530 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:10.966466    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podStartSLOduration=1.966447362 podStartE2EDuration="1.966447362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:10.962724249 +0000 UTC m=+2339.077458439" watchObservedRunningTime="2026-10-30 21:17:10.966447362 +0000 UTC m=+2339.081181553"
Oct 30 21:17:11.278936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:11.278902    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:11.280535 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:11.280507    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:11.280673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:11.280577    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:11.483246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:11.483187    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:12.279030 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:12.278992    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:12.279582 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:12.279055    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:12.409780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:12.409731    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:13.293413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:13.292653    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:13.293413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:13.292715    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:14.286411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:14.285869    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:14.286411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:14.285941    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:14.477413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:14.476948    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:15.279803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:15.279761    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:15.280520 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:15.280077    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:15.539756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:15.539448    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:16.285856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:16.285816    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:16.286159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:16.286127    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:16.626045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:16.625928    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:16.652162 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:16.652126    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:16.656829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:16.656789    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:16.658024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:16.657756    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.241707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.239570    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.282115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.282059    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:17.292983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.292932    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:17.365412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.364557    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.403410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.394433    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.628648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.628537    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:17.658928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.658877    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.659579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.659550    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 21:17:17.660533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.660082    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.660533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.660340    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:17:17.660533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.660420    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:17:17.685217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.679485    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.702131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.700744    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:17.878325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.878296    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:17:17.879974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:17.879899    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:17:18.279611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:18.279577    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:18.279803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:18.279641    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:18.622377 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:18.622279    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:19.278998 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:19.278961    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 30 21:17:19.279597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:19.279040    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 30 21:17:19.557095 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:19.556999    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:20.353706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:20.353666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:20.395476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:20.395442    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:20.395991 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:20.395972    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:20.396833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:20.396809    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:17:21.229004 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:21.228965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:21.568030 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:17:21.567987    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:17:22.065118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:22.065081    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:23.038793 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:23.038732    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:23.844741 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:23.844710    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:24.625358 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:24.625323    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:25.316077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:25.316043    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:26.137684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:26.137646    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:26.939653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:26.939614    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:27.738193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:27.738152    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:28.513252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:28.513209    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:29.344121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:29.344063    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:30.226751 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:30.226689    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:31.087378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:31.087337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:31.849802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:31.849759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:32.629351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:32.629310    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:33.346659 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:33.346601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:34.041689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:34.041657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:34.917450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:34.917415    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:35.618201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:35.618166    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:36.326046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:36.326010    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:37.111935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:37.111900    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:37.909778 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:37.909725    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:38.605153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:38.605123    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:39.574928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:39.574883    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:40.456187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:40.456154    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:41.243688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:41.243654    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:42.250936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:42.250898    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:43.115349 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:43.115272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:43.917619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:43.917584    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:44.869822 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:44.869790    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:45.808327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:45.808288    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:46.718917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:46.718883    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:47.617785 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:47.617755    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:48.426076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:48.426038    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:49.326557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:49.326518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:49.737596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:49.737508    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"]
Oct 30 21:17:49.737762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:49.737722    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy" containerID="cri-o://777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00" gracePeriod=30
Oct 30 21:17:50.170519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:50.170468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:51.148838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:51.148807    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:51.562258 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:17:51.562212    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:17:52.053621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:52.053591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:52.841019 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:52.840987    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:53.632672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:53.632638    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:54.389128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:54.389087    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:55.023255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.023189    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 21:17:55.113586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.113546    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-credential-socket\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.113787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.113699    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-istio-token\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-istio-token\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.113787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.113773    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-kube-api-access-kb6vj\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-kube-api-access-kb6vj\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.113806    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6db7f461-eb75-4c63-960a-ed412a088428-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db7f461-eb75-4c63-960a-ed412a088428-istio-podinfo\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.113873    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-credential-socket" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.113977    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-socket\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.114036    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-data\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.114090    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6db7f461-eb75-4c63-960a-ed412a088428-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db7f461-eb75-4c63-960a-ed412a088428-istiod-ca-cert\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.114120    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-envoy\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.114170    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-certs\") pod \"6db7f461-eb75-4c63-960a-ed412a088428\" (UID: \"6db7f461-eb75-4c63-960a-ed412a088428\") "
Oct 30 21:17:55.115075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.114929    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-data" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:55.115556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.115318    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-socket" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:55.115763 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.115738    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.115853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.115770    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.115853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.115787    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.116022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.116000    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-certs" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:55.116716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.116687    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-kube-api-access-kb6vj" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "kube-api-access-kb6vj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:55.117796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.117772    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6db7f461-eb75-4c63-960a-ed412a088428-istiod-ca-cert" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:17:55.119897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.118026    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6db7f461-eb75-4c63-960a-ed412a088428-istio-podinfo" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:17:55.120201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.120177    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-istio-token" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:17:55.125856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.125833    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-envoy" pod "6db7f461-eb75-4c63-960a-ed412a088428" (UID: "6db7f461-eb75-4c63-960a-ed412a088428"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:17:55.152452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.151115    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:55.239411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.233072    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.239411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.233116    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb6vj\" (UniqueName: \"kubernetes.io/projected/6db7f461-eb75-4c63-960a-ed412a088428-kube-api-access-kb6vj\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.239411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.233132    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6db7f461-eb75-4c63-960a-ed412a088428-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.239411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.233147    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6db7f461-eb75-4c63-960a-ed412a088428-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.239411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.233161    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.239411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.233174    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6db7f461-eb75-4c63-960a-ed412a088428-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:17:55.789413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.788983    4852 generic.go:356] "Generic (PLEG): container finished" podID="6db7f461-eb75-4c63-960a-ed412a088428" containerID="777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00" exitCode=0
Oct 30 21:17:55.789413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.789150    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" event={"ID":"6db7f461-eb75-4c63-960a-ed412a088428","Type":"ContainerDied","Data":"777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00"}
Oct 30 21:17:55.789413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.789201    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw" event={"ID":"6db7f461-eb75-4c63-960a-ed412a088428","Type":"ContainerDied","Data":"903a7866d6a5ab325e3821eba5059da6add8bc25ab5e954e2be25b1276d03751"}
Oct 30 21:17:55.789413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.789234    4852 scope.go:122] "RemoveContainer" containerID="777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00"
Oct 30 21:17:55.790073 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.789453    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"
Oct 30 21:17:55.914434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.913601    4852 scope.go:122] "RemoveContainer" containerID="777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00"
Oct 30 21:17:55.914749 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:17:55.914643    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00\": container with ID starting with 777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00 not found: ID does not exist" containerID="777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00"
Oct 30 21:17:55.914749 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.914686    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00"} err="failed to get container status \"777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00\": rpc error: code = NotFound desc = could not find container \"777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00\": container with ID starting with 777cdab84e1049d32f61f5f59f6267492a0a00d1974a6139b023909a72d07c00 not found: ID does not exist"
Oct 30 21:17:55.943617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.943581    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"]
Oct 30 21:17:55.951859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:55.951823    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-2wvpw"]
Oct 30 21:17:56.014912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:56.014868    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6db7f461-eb75-4c63-960a-ed412a088428" path="/var/lib/kubelet/pods/6db7f461-eb75-4c63-960a-ed412a088428/volumes"
Oct 30 21:17:56.989422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:56.980815    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:57.980412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:57.974909    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:58.360411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.360259    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:58.403411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.400845    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:17:58.651471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.649500    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 21:17:58.671419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.668281    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:17:58.671419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.669462    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:58.671419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.670562    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:58.903619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.903505    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:58.904307 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:58.904272    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 21:17:59.078950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:59.078870    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:59.333972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:59.333933    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:59.428190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:17:59.428156    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:00.111288 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:00.111252    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:00.745771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:00.745738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:01.516153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:01.516115    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:02.236108 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:02.236073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:02.937962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:02.937931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:03.720517 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:03.720481    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:04.524571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:04.524539    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:04.948313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:04.948226    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:05.295850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:05.295815    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:06.254500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:06.254466    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.026818 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:07.026784    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.709328 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:07.709286    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:08.429239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:08.429205    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:09.224719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:09.224680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:09.942989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:09.942956    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:10.800522 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:10.800490    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:11.506304 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:11.506276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:12.230135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:12.230093    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:12.936172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:12.936140    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:13.658554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:13.658520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.541627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:14.541591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:15.258823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:15.258784    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:16.041981 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:16.041939    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:16.822627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:16.822589    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:17.567806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:17.567765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:18.434353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:18.434318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:19.200444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:19.200375    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:19.911580 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:19.911538    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:20.737695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:20.737661    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:21.528866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:21.528827    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:21.559269 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:18:21.559230    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:18:22.407960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:22.407909    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:23.570678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:23.570639    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:24.431275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:24.431224    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:25.404966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:25.404932    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:26.139955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:26.139919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:26.856098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:26.856063    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:27.629309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:27.629272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:28.445349 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:28.445317    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:29.359304 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:29.359261    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:30.145652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:30.145610    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:30.899836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:30.899807    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:31.450661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:31.450625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:31.529506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:31.529465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:31.609086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:31.609054    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:32.417638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:32.417588    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:33.226907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:33.226873    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:33.940209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:33.940179    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:34.704859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:34.704819    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:35.445958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:35.445921    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:36.235365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:36.235329    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:37.106682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:37.106619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:37.183451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:37.183420    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:37.745505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:37.745469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:38.150505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:38.150461    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:38.982180 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:38.982149    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:39.715795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:39.715761    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:40.414048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:40.414017    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:41.148580 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:41.148544    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:41.898536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:41.898502    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:42.605524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:42.605487    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:43.342327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:43.342296    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:44.188435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:44.188404    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:44.299354 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:44.298261    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:44.389544 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:44.389503    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:44.913418 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:44.913357    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:45.736539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:45.736505    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:46.612852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:46.612818    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:47.409006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:47.408964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:48.139783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:48.139749    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:48.845025 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:48.844986    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:49.535122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:49.535090    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:50.248538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:50.248470    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:50.560706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:50.560663    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:50.647868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:50.647834    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:51.128041 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:51.128001    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:51.460571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:51.460473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:51.559445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:51.559351    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:51.564703 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:18:51.564669    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:18:52.621643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:52.621613    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:53.032411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:53.029473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:53.155418 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:53.139093    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:54.115944 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:54.115897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:55.028749 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:55.028695    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:55.926569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:55.926529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.113077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:56.113042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.384607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:56.384557    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.482832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:56.482771    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.682716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:56.682600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.888642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:56.888606    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:57.257888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:57.257851    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:57.357539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:57.357504    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:58.430071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:58.430032    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:58.557447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:58.554841    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:58.962885 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:18:58.962847    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:00.124758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:00.124717    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:00.930094 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:00.930058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:01.610090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:01.610055    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:02.338234 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:02.338198    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:03.021042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:03.021009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:03.725892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:03.725855    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:04.434342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:04.434308    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:04.990778 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:04.990738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:05.273648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:05.273618    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:06.206414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:06.205136    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:06.973281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:06.973240    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:07.692658 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:07.692625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:08.521923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:08.521882    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:09.336588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:09.336555    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:10.231618 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:10.231581    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:10.912758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:10.912721    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:11.715661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:11.715630    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:12.514469 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:12.514422    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:13.324944 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:13.324904    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:14.257410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:14.244412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:15.211921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:15.211869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:15.533593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:15.533556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:16.023632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:16.023601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:16.725507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:16.725469    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:17.538265 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:17.538223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:18.243639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:18.243605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.035015 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:19.034973    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.824988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:19.824951    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:20.628602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:20.628568    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:21.341504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:21.341457    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:21.561024 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:19:21.560982    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:19:22.115910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:22.115874    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:22.842084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:22.842049    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:23.710277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:23.710245    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:24.414117 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:24.414078    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:25.235184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:25.235149    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:25.918519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:25.918480    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:26.616311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:26.616275    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:27.342949 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:27.342905    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:28.095989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:28.095956    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:28.781968 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:28.781928    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:29.531998 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:29.531964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.311397 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:30.311365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.981620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:30.981291    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:31.072975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:31.072934    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:31.173672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:31.173631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:32.312589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:32.312552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:33.029625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:33.029591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:33.830634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:33.830600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:34.529661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:34.529629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:35.240811 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:35.240776    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:36.006416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:36.006369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:36.708879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:36.708843    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:37.435075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:37.435039    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:38.151700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:38.151666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:39.020954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:39.020917    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:39.733051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:39.733012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:40.469634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:40.469596    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:41.232774 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:41.232745    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:42.026591 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:42.026556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:42.622193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:42.622156    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:42.715546 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:42.715509    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:43.253621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:43.253582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:43.354629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:43.354574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:44.239596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:44.239556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.019491 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:45.019455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.840689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:45.840657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:46.558245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:46.558211    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:47.535403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:47.535353    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:48.222955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:48.222925    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:48.908695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:48.908660    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:49.644515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:49.644482    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:49.751566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:49.751536    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:49.844490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:49.844458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:50.123409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:50.123244    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:50.376227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:50.376132    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:50.721067 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:50.720988    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:50.977297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:50.977203    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:51.083593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:51.083560    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:51.560061 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:19:51.560026    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:19:52.185975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:52.185938    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:52.356339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:52.356302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:52.965154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:52.965119    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:53.712308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:53.712274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:54.516130 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:54.516090    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.207823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:55.207793    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.931425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:55.931359    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:56.712557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:56.712519    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:57.489914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:57.489873    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:57.715260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:57.715200    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:57.815711 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:57.815669    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:59.236076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:19:59.236046    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:00.019214 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:00.019181    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:00.802638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:00.802607    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.514000 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:01.513958    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:02.234099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:02.234060    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:03.039587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:03.039547    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:03.726983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:03.726950    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:04.417046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:04.416982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:05.046359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:05.046312    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:05.305449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:05.305326    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:06.135617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:06.135577    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:06.796790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:06.796748    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"]
Oct 30 21:20:06.797058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:06.797010    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy" containerID="cri-o://76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b" gracePeriod=30
Oct 30 21:20:06.802497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:06.802014    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"]
Oct 30 21:20:06.802497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:06.802251    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy" containerID="cri-o://fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c" gracePeriod=30
Oct 30 21:20:06.914536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:06.914497    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:07.843291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:07.843259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:08.553576 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:08.553539    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:09.252483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:09.252438    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:10.054717 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:10.054647    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:10.821358 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:10.821297    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:11.521171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:11.521133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:12.112221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.112183    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:20:12.187072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.186966    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:20:12.255504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255470    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-socket\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.255504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255515    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0975483-3126-4e3a-8cf5-9eba54e64198-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0975483-3126-4e3a-8cf5-9eba54e64198-istiod-ca-cert\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.255752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255542    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-podinfo\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.255752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255592    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-credential-socket\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.255752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255623    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istiod-ca-cert\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.255752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255652    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-certs\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.255752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255678    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-kube-api-access-fpln8\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-kube-api-access-fpln8\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.255932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255825    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-podinfo\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.255932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255826    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-socket" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.255932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255877    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-token\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-token\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.255932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255910    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-envoy\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.256062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255941    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-certs\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.256062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.255978    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-kube-api-access-rcvqd\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-kube-api-access-rcvqd\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.256062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256021    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-data\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.256158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256077    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-credential-socket" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.256158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256090    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-socket\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.256158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256120    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-token\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-token\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.256259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256160    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-envoy\") pod \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\" (UID: \"618802d8-a3bc-4e09-846a-b3fd0ca1327c\") "
Oct 30 21:20:12.256259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256189    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-data\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.256259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256212    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-credential-socket\") pod \"d0975483-3126-4e3a-8cf5-9eba54e64198\" (UID: \"d0975483-3126-4e3a-8cf5-9eba54e64198\") "
Oct 30 21:20:12.256432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256411    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0975483-3126-4e3a-8cf5-9eba54e64198-istiod-ca-cert" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:20:12.256884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256860    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.256990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256889    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0975483-3126-4e3a-8cf5-9eba54e64198-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.256990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256904    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.256990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.256926    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-certs" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.257226 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.257121    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-credential-socket" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.257325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.257305    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-data" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.257953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.257915    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-socket" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.257953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.257926    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-certs" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.258154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.258105    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-data" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.258467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.258217    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istiod-ca-cert" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:20:12.259729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.259702    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-token" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:20:12.260217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.260190    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-envoy" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.260298 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.260220    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-podinfo" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:20:12.260604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.260578    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-token" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:20:12.261319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.261280    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-envoy" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:12.261647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.261621    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-kube-api-access-rcvqd" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "kube-api-access-rcvqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:20:12.261759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.261739    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-podinfo" pod "618802d8-a3bc-4e09-846a-b3fd0ca1327c" (UID: "618802d8-a3bc-4e09-846a-b3fd0ca1327c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:20:12.261851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.261835    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-kube-api-access-fpln8" pod "d0975483-3126-4e3a-8cf5-9eba54e64198" (UID: "d0975483-3126-4e3a-8cf5-9eba54e64198"). InnerVolumeSpecName "kube-api-access-fpln8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:20:12.358373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358270    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358310    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358325    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358334    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358344    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcvqd\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-kube-api-access-rcvqd\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358354    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358368    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358381    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358413    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358425    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358434    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358444    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0975483-3126-4e3a-8cf5-9eba54e64198-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358458    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/618802d8-a3bc-4e09-846a-b3fd0ca1327c-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358466    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0975483-3126-4e3a-8cf5-9eba54e64198-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.358787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.358475    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpln8\" (UniqueName: \"kubernetes.io/projected/d0975483-3126-4e3a-8cf5-9eba54e64198-kube-api-access-fpln8\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:12.424526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.424302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:12.975852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.974592    4852 generic.go:356] "Generic (PLEG): container finished" podID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerID="76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b" exitCode=0
Oct 30 21:20:12.975852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.974713    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"
Oct 30 21:20:12.975852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.974855    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" event={"ID":"d0975483-3126-4e3a-8cf5-9eba54e64198","Type":"ContainerDied","Data":"76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b"}
Oct 30 21:20:12.975852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.974900    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v" event={"ID":"d0975483-3126-4e3a-8cf5-9eba54e64198","Type":"ContainerDied","Data":"e479096af085059bbcc7aab57ee5abcf417834c53d8e662185df8bb819f05773"}
Oct 30 21:20:12.975852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.974927    4852 scope.go:122] "RemoveContainer" containerID="76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b"
Oct 30 21:20:12.982449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.982357    4852 generic.go:356] "Generic (PLEG): container finished" podID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerID="fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c" exitCode=0
Oct 30 21:20:12.982449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.982447    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" event={"ID":"618802d8-a3bc-4e09-846a-b3fd0ca1327c","Type":"ContainerDied","Data":"fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c"}
Oct 30 21:20:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.982517    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h" event={"ID":"618802d8-a3bc-4e09-846a-b3fd0ca1327c","Type":"ContainerDied","Data":"c98ead870e124dc48ad2be1f8b48360b01a6cafca1252ee01b9117a5260a87a1"}
Oct 30 21:20:12.982675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:12.982619    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"
Oct 30 21:20:13.033081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.032950    4852 scope.go:122] "RemoveContainer" containerID="76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b"
Oct 30 21:20:13.033764 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:20:13.033709    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b\": container with ID starting with 76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b not found: ID does not exist" containerID="76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b"
Oct 30 21:20:13.033894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.033755    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b"} err="failed to get container status \"76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b\": rpc error: code = NotFound desc = could not find container \"76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b\": container with ID starting with 76bbfe7e3c13eda2134ab563969a131a910d12fb562eda520007a01ec4d61a0b not found: ID does not exist"
Oct 30 21:20:13.033894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.033784    4852 scope.go:122] "RemoveContainer" containerID="fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c"
Oct 30 21:20:13.052559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.052524    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"]
Oct 30 21:20:13.056347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.056317    4852 scope.go:122] "RemoveContainer" containerID="075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e"
Oct 30 21:20:13.058277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.058251    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-2sp8v"]
Oct 30 21:20:13.078145 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.078117    4852 scope.go:122] "RemoveContainer" containerID="fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c"
Oct 30 21:20:13.079011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.078985    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"]
Oct 30 21:20:13.080369 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:20:13.080342    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c\": container with ID starting with fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c not found: ID does not exist" containerID="fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c"
Oct 30 21:20:13.080576 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.080544    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c"} err="failed to get container status \"fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c\": rpc error: code = NotFound desc = could not find container \"fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c\": container with ID starting with fc7e3761a18a640f6398048c21b52c6b24e359b503b0390bfb6b313f9175c41c not found: ID does not exist"
Oct 30 21:20:13.080710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.080696    4852 scope.go:122] "RemoveContainer" containerID="075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e"
Oct 30 21:20:13.082127 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:20:13.082065    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e\": container with ID starting with 075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e not found: ID does not exist" containerID="075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e"
Oct 30 21:20:13.082127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.082097    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e"} err="failed to get container status \"075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e\": rpc error: code = NotFound desc = could not find container \"075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e\": container with ID starting with 075a12ae87c6b069f82049913fd3f4eaffa59669058652272a1ee9f74ccc615e not found: ID does not exist"
Oct 30 21:20:13.087728 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.087687    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-sgx2h"]
Oct 30 21:20:13.721054 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:13.721019    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:14.012479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:14.012439    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" path="/var/lib/kubelet/pods/618802d8-a3bc-4e09-846a-b3fd0ca1327c/volumes"
Oct 30 21:20:14.013132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:14.013109    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" path="/var/lib/kubelet/pods/d0975483-3126-4e3a-8cf5-9eba54e64198/volumes"
Oct 30 21:20:14.512170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:14.512124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:15.219565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:15.219529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:16.019152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:16.019114    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:16.715312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:16.715267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:17.422662 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:17.422619    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:18.136801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:18.136612    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:18.902942 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:18.902905    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:19.707848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:19.707816    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:20.436086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:20.436051    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:21.202894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:21.202859    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:21.559794 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:20:21.559754    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:20:21.981478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:21.981367    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:22.735462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:22.735368    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:23.424592 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:23.424556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.123974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:24.123933    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.810718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:24.810679    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:25.615504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:25.615468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:26.340248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:26.340211    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:27.098130 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:27.098081    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:27.943284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:27.943249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:28.705533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:28.705498    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:29.417922 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:29.417888    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:30.225844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:30.225802    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:31.024604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:31.024567    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:31.821713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:31.821667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:32.690694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:32.690659    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:33.332049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:33.332012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:34.107961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:34.107930    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:34.880712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:34.880678    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:35.557531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:35.557466    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:36.405403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:36.405354    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:37.112293 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:37.112243    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:37.838189 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:37.838144    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:38.545607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:38.545574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:39.311102 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:39.311067    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:40.069199 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:40.069151    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:40.941678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:40.941627    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:41.754716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:41.754667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:42.520340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:42.520270    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:43.307489 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:43.307451    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:44.016882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:44.016851    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:44.792416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:44.792357    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:45.516554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:45.516518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:46.218309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:46.218233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:47.038567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:47.038532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:47.782814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:47.782774    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:48.500489 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:48.500451    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:49.228726 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:49.228674    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:50.012001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:50.011959    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:50.702790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:50.702754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:51.442546 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:51.442501    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:51.569806 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:20:51.569764    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:20:52.287220 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:52.287172    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:53.023035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:53.022981    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:53.742555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:53.742510    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:54.471821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:54.471789    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:55.326192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:55.326155    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:56.007292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:56.007258    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:56.734734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:56.734687    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:57.527287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:57.527241    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.300601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:58.300569    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.977287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:58.977246    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:59.735345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:20:59.735312    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:00.429092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:00.429053    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:01.108631 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:01.108587    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:01.809305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:01.809266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:02.637184 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:02.637145    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:02.908923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:02.908889    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:03.005242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:03.005206    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:03.343874 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:03.343836    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:04.036281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:04.036228    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:04.826571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:04.826535    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.093452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:05.093328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.607080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:05.607040    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:06.464699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:06.464658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:07.406051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:07.406009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:08.111651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:08.111616    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:08.859121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:08.859091    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:09.732691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:09.732654    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:10.443946 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:10.443869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:11.138879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:11.138839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:11.904275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:11.904212    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:12.607635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:12.607596    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:13.317068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:13.317037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:14.096806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:14.096758    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:14.819141 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:14.819098    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:15.529875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:15.529824    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:16.276822 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:16.276790    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:17.132323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:17.132282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:17.815787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:17.815737    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:18.520864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:18.520829    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.257492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:19.257452    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:20.005140 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:20.005103    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.057158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:21.057123    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.560529 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:21:21.560495    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:21:21.886546 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:21.886462    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:22.586796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:22.586761    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:23.015720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:23.015679    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:23.120904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:23.120857    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:24.009032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:24.008994    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:24.915416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:24.915339    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:25.644310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:25.644275    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:26.460798 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:26.460756    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:27.299254 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:27.299220    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:28.011938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:28.011896    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:28.737268 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:28.737233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:29.492791 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:29.492754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:30.211468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:30.211434    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:30.909245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:30.909214    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:31.758787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:31.758745    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:32.504728 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:32.504693    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:33.216168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:33.216133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:33.950252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:33.950220    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:34.714891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:34.714853    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:35.408975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:35.408940    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:36.275660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:36.275625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:37.579047 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:37.579009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:38.398236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:38.398201    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:39.152939 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:39.152905    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:39.860403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:39.860363    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:40.626885 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:40.626854    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:41.382447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:41.382376    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:42.145212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:42.145177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:43.004564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:43.004531    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:43.718279 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:43.718243    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.492806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:44.492771    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:45.209041 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:45.208998    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:45.939844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:45.939807    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:46.624279 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:46.624208    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:47.337302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:47.337269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:48.040541 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:48.040505    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:48.732740 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:48.732705    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:49.471272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:49.471235    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:50.325715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:50.325683    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:51.020301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:51.020257    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:51.559592 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:21:51.559555    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:21:51.738004 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:51.737967    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:52.439933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:52.439897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:53.296852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:53.296817    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:54.002888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:54.002850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:54.746432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:54.746371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:55.422956 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:55.422924    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:56.108483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:56.108442    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:56.840121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:56.840086    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:57.607674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:57.607643    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:58.311545 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:58.311511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.055040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:59.055006    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.822552 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:21:59.822520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:00.506510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:00.506474    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"]
Oct 30 21:22:00.506758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:00.506707    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy" containerID="cri-o://af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982" gracePeriod=30
Oct 30 21:22:00.813483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:00.813396    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:00.912842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:00.912813    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.174309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:02.174267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.613858 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:02.613826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.715701 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:02.715667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.898236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:02.898207    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:03.080771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:03.080733    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:04.201649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:04.201612    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:04.630064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:04.630032    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:05.041317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.041282    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"]
Oct 30 21:22:05.041560 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.041527    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy" containerID="cri-o://35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7" gracePeriod=30
Oct 30 21:22:05.158439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.158372    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:05.323005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.322924    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:05.798839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.798809    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 21:22:05.844299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844265    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-certs\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.844514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844340    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-data\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.844514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844399    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-socket\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.844514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844489    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-credential-socket\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.844710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844520    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-kube-api-access-8hc8l\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-kube-api-access-8hc8l\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.844710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844612    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-certs" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:05.844824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844743    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-socket" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:05.844824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844750    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-data" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:05.844928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844627    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-token\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-token\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.844990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844936    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-credential-socket" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:05.844990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.844942    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-envoy\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.845140 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.845115    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-podinfo\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.845269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.845251    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istiod-ca-cert\") pod \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\" (UID: \"c925dcc8-57c5-47aa-9dd4-a011fadf74da\") "
Oct 30 21:22:05.846204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.845674    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istiod-ca-cert" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:05.846204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.846105    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.846204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.846130    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.846204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.846146    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.846204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.846160    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.846204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.846174    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.847612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.847591    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-kube-api-access-8hc8l" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "kube-api-access-8hc8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:05.847735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.847628    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-token" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:05.847883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.847853    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-podinfo" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:05.848864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.848845    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-envoy" pod "c925dcc8-57c5-47aa-9dd4-a011fadf74da" (UID: "c925dcc8-57c5-47aa-9dd4-a011fadf74da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:05.948091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.948003    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hc8l\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-kube-api-access-8hc8l\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.948091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.948034    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.948091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.948043    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:05.948091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:05.948054    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c925dcc8-57c5-47aa-9dd4-a011fadf74da-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:06.012400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.012348    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:06.429435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.429365    4852 generic.go:356] "Generic (PLEG): container finished" podID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerID="af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982" exitCode=0
Oct 30 21:22:06.429840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.429736    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" event={"ID":"c925dcc8-57c5-47aa-9dd4-a011fadf74da","Type":"ContainerDied","Data":"af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982"}
Oct 30 21:22:06.429840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.429796    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8" event={"ID":"c925dcc8-57c5-47aa-9dd4-a011fadf74da","Type":"ContainerDied","Data":"5cdb139b0a34d7624f0c087128c517bdf6ef5ad33f5afc9f12f2b5437a223e02"}
Oct 30 21:22:06.429840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.429826    4852 scope.go:122] "RemoveContainer" containerID="af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982"
Oct 30 21:22:06.430027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.430012    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"
Oct 30 21:22:06.449935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.449913    4852 scope.go:122] "RemoveContainer" containerID="2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1"
Oct 30 21:22:06.463878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.463852    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"]
Oct 30 21:22:06.464120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.464100    4852 scope.go:122] "RemoveContainer" containerID="af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982"
Oct 30 21:22:06.464503 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:06.464482    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982\": container with ID starting with af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982 not found: ID does not exist" containerID="af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982"
Oct 30 21:22:06.464560 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.464514    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982"} err="failed to get container status \"af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982\": rpc error: code = NotFound desc = could not find container \"af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982\": container with ID starting with af19ac72ffc29b6244dae8ef5afa1523052880efc2c813d68a08ead9464e8982 not found: ID does not exist"
Oct 30 21:22:06.464560 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.464533    4852 scope.go:122] "RemoveContainer" containerID="2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1"
Oct 30 21:22:06.464844 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:06.464821    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1\": container with ID starting with 2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1 not found: ID does not exist" containerID="2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1"
Oct 30 21:22:06.464907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.464857    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1"} err="failed to get container status \"2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1\": rpc error: code = NotFound desc = could not find container \"2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1\": container with ID starting with 2c82631f6d849f0dbb680ef890a51bd398901973ecb5ec710561e154f6b66fe1 not found: ID does not exist"
Oct 30 21:22:06.468635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.468607    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-gtbz8"]
Oct 30 21:22:06.840312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:06.840273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:07.732720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:07.732684    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:08.006771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:08.006744    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" path="/var/lib/kubelet/pods/c925dcc8-57c5-47aa-9dd4-a011fadf74da/volumes"
Oct 30 21:22:08.424465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:08.424364    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:09.149338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:09.149306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:09.845833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:09.845800    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:10.340225 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.340200    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 21:22:10.477996 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.477634    4852 generic.go:356] "Generic (PLEG): container finished" podID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerID="35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7" exitCode=0
Oct 30 21:22:10.477996 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.477964    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"
Oct 30 21:22:10.478485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.478453    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" event={"ID":"cd1dd97b-0870-4c4d-b313-685d43285e96","Type":"ContainerDied","Data":"35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7"}
Oct 30 21:22:10.478694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.478530    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw" event={"ID":"cd1dd97b-0870-4c4d-b313-685d43285e96","Type":"ContainerDied","Data":"09360bcbaffa755929527a5b5e0f7f5e7088fa1b41eccaea9fa72b751c89833b"}
Oct 30 21:22:10.478694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.478564    4852 scope.go:122] "RemoveContainer" containerID="35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7"
Oct 30 21:22:10.490325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490295    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-podinfo\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.490512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490337    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-credential-socket\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.490512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490475    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-data\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.490614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490510    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-kube-api-access-cgk9w\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-kube-api-access-cgk9w\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.490721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490692    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-certs\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.490808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490741    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-token\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-token\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.490808 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490791    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd1dd97b-0870-4c4d-b313-685d43285e96-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1dd97b-0870-4c4d-b313-685d43285e96-istiod-ca-cert\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.491008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490823    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-credential-socket" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:10.491008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490832    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-data" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:10.491008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490889    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-socket\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.491008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.490959    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-envoy\") pod \"cd1dd97b-0870-4c4d-b313-685d43285e96\" (UID: \"cd1dd97b-0870-4c4d-b313-685d43285e96\") "
Oct 30 21:22:10.491525 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491229    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd1dd97b-0870-4c4d-b313-685d43285e96-istiod-ca-cert" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:10.491525 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491234    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-certs" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:10.491525 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491376    4852 scope.go:122] "RemoveContainer" containerID="35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7"
Oct 30 21:22:10.491525 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491481    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-socket" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:10.491854 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:10.491831    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7\": container with ID starting with 35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7 not found: ID does not exist" containerID="35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7"
Oct 30 21:22:10.491947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491854    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.491947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491876    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.491947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491866    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7"} err="failed to get container status \"35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7\": rpc error: code = NotFound desc = could not find container \"35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7\": container with ID starting with 35a329ef0ee01058b6375fde46b96304d7fbfa66f08701a36d14d2a665f801b7 not found: ID does not exist"
Oct 30 21:22:10.491947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491891    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd1dd97b-0870-4c4d-b313-685d43285e96-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.491947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491907    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.491947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.491920    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.492843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.492811    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-podinfo" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:10.493098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.493079    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-kube-api-access-cgk9w" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "kube-api-access-cgk9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:10.493376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.493357    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-token" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:10.493596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.493577    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-envoy" pod "cd1dd97b-0870-4c4d-b313-685d43285e96" (UID: "cd1dd97b-0870-4c4d-b313-685d43285e96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:10.593098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.593064    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.593098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.593093    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgk9w\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-kube-api-access-cgk9w\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.593098 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.593105    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.593331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.593114    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd1dd97b-0870-4c4d-b313-685d43285e96-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:10.699435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.699370    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:10.858418 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.858353    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"]
Oct 30 21:22:10.864694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:10.864666    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-9fzqw"]
Oct 30 21:22:11.406649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:11.406608    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.008175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:12.008136    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" path="/var/lib/kubelet/pods/cd1dd97b-0870-4c4d-b313-685d43285e96/volumes"
Oct 30 21:22:12.135407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:12.135362    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"]
Oct 30 21:22:12.135678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:12.135633    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy" containerID="cri-o://2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687" gracePeriod=30
Oct 30 21:22:12.155176 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:12.155137    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:13.075456 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:13.075362    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:13.830306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:13.830274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:14.714504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:14.714415    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:14.809638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:14.809608    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:14.922275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:14.922237    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:15.956541 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:15.956510    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:16.124113 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:16.124077    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:17.220827 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:17.220788    4852 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642feace_727f_49a7_8955_b895832d2540.slice/crio-2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:22:17.251078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.251036    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:17.478499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.478467    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 21:22:17.565479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.565437    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/642feace-727f-49a7-8955-b895832d2540-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642feace-727f-49a7-8955-b895832d2540-istio-podinfo\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.565661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.565567    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-certs\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.565661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.565648    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-credential-socket\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.565799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.565735    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-envoy\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.565799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.565766    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-data\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.565911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.565848    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-istio-token\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-istio-token\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.566051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.566031    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/642feace-727f-49a7-8955-b895832d2540-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642feace-727f-49a7-8955-b895832d2540-istiod-ca-cert\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.566194 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.566160    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-credential-socket" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:17.566259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.566175    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-kube-api-access-tbnnn\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.566317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.566262    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-socket\") pod \"642feace-727f-49a7-8955-b895832d2540\" (UID: \"642feace-727f-49a7-8955-b895832d2540\") "
Oct 30 21:22:17.567105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.567075    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.567340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.567316    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-socket" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:17.567554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.567531    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-certs" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:17.567741 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.567718    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-data" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:17.569028 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.568995    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-kube-api-access-tbnnn" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "kube-api-access-tbnnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:17.570401 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.570358    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/642feace-727f-49a7-8955-b895832d2540-istio-podinfo" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:22:17.570815 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.570766    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/642feace-727f-49a7-8955-b895832d2540-istiod-ca-cert" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:17.575443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.575370    4852 generic.go:356] "Generic (PLEG): container finished" podID="642feace-727f-49a7-8955-b895832d2540" containerID="2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687" exitCode=0
Oct 30 21:22:17.575443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.575436    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" event={"ID":"642feace-727f-49a7-8955-b895832d2540","Type":"ContainerDied","Data":"2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687"}
Oct 30 21:22:17.575580 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.575486    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8" event={"ID":"642feace-727f-49a7-8955-b895832d2540","Type":"ContainerDied","Data":"b377e09643ec45791c55459bdd815ea96dec80a746b3b1062dd2603e69f07449"}
Oct 30 21:22:17.575580 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.575508    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"
Oct 30 21:22:17.575580 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.575517    4852 scope.go:122] "RemoveContainer" containerID="2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687"
Oct 30 21:22:17.581796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.581765    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-istio-token" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:17.581897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.581772    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-envoy" pod "642feace-727f-49a7-8955-b895832d2540" (UID: "642feace-727f-49a7-8955-b895832d2540"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:22:17.607193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.607163    4852 scope.go:122] "RemoveContainer" containerID="836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447"
Oct 30 21:22:17.632695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.632666    4852 scope.go:122] "RemoveContainer" containerID="2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687"
Oct 30 21:22:17.633058 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:17.633032    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687\": container with ID starting with 2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687 not found: ID does not exist" containerID="2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687"
Oct 30 21:22:17.633178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.633070    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687"} err="failed to get container status \"2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687\": rpc error: code = NotFound desc = could not find container \"2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687\": container with ID starting with 2eaf97777ef0d240432ac2d64d1760a6c3a2d01fde804fb32df62b73decae687 not found: ID does not exist"
Oct 30 21:22:17.633178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.633095    4852 scope.go:122] "RemoveContainer" containerID="836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447"
Oct 30 21:22:17.633474 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:17.633446    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447\": container with ID starting with 836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447 not found: ID does not exist" containerID="836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447"
Oct 30 21:22:17.633594 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.633485    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447"} err="failed to get container status \"836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447\": rpc error: code = NotFound desc = could not find container \"836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447\": container with ID starting with 836236a4cdc2284c4a36c717b01fc2a6cfd1f42758eb13f7150bbea3a4e1c447 not found: ID does not exist"
Oct 30 21:22:17.668096 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668067    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.668096 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668096    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.668297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668110    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.668297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668122    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.668297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668134    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642feace-727f-49a7-8955-b895832d2540-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.668297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668148    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/642feace-727f-49a7-8955-b895832d2540-kube-api-access-tbnnn\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.668297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668162    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642feace-727f-49a7-8955-b895832d2540-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.668297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.668175    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642feace-727f-49a7-8955-b895832d2540-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:17.902841 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.902805    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"]
Oct 30 21:22:17.907051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:17.907022    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-xhhr8"]
Oct 30 21:22:18.006776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:18.006746    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="642feace-727f-49a7-8955-b895832d2540" path="/var/lib/kubelet/pods/642feace-727f-49a7-8955-b895832d2540/volumes"
Oct 30 21:22:18.021968 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:18.021943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:18.720577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:18.720545    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:19.530403 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:19.530365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:20.313780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:20.313744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:21.035034 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:21.034998    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:21.558535 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:21.558500    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:22:21.736560 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:21.736521    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:22.405374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:22.405338    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:23.144266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:23.144228    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:23.831065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:23.831031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:24.759011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:24.758966    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:25.520794 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:25.520756    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.591039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:26.590999    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:27.571187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:27.571062    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:28.241733 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:28.241683    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:29.032799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:29.032765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:29.931538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:29.928027    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:30.724363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:30.724324    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:31.642624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:31.642591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:32.422917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:32.422881    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:33.129547 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:33.129514    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:33.858236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:33.858199    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:34.631338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:34.631303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:35.485614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:35.485581    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:36.167634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:36.167596    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:36.895972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:36.895935    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:37.625821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:37.625782    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.318350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:38.318311    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:39.025196 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:39.025159    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:39.731782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:39.731750    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:40.430889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:40.430853    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:41.114256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:41.114223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:41.917730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:41.917700    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:43.354444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:43.354373    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:44.044111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:44.044073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:44.707799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:44.707764    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:45.442906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:45.442873    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:46.203442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:46.203366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:46.424289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:46.424249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:46.523579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:46.523540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:47.891789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:47.891738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:48.207806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:48.207718    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:48.307154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:48.307125    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:49.611336 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:49.611295    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:50.174727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:50.174684    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:50.302408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:50.297404    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:50.804285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:50.804251    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:51.561579 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:22:51.561540    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:22:51.830787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:51.830709    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:52.544753 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:52.544696    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:53.310923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:53.310865    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:54.129775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:54.129737    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:54.670875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:54.670837    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:54.805578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:54.805531    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:55.333316 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:55.333266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:56.329443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:56.329365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:57.145285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:57.145255    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:57.870297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:57.870254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:58.626879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:58.626846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:59.412521 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:22:59.412478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:00.232120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:00.232078    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:00.969680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:00.969643    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:01.712299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:01.712264    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:02.517734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:02.517700    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:03.241061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:03.240953    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:04.313931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:04.313890    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:05.115373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:05.115333    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:05.231518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:05.231474    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:05.914482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:05.914444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:06.612466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:06.612439    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:07.408131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:07.408098    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:08.228921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:08.228876    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:08.369680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:08.369639    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:08.473157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:08.473126    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:08.936637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:08.936602    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:09.749586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:09.749547    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.423011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:10.422979    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.733011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:10.732919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.842411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:10.841071    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:11.944213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:11.944179    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:12.323904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:12.323865    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:12.416809 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:12.416775    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:12.804422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:12.803601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:13.101260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:13.101176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:13.201745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:13.201705    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:13.306301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:13.306268    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:13.563898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:13.563857    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:13.695867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:13.695834    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.597865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:14.597831    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.687398 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:14.687295    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.829908 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:14.829873    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.921600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:14.921520    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:15.215891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:15.215839    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:15.427039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:15.426958    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:15.605355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:15.605287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:16.157643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:16.157606    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.012551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:17.012514    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.381732 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:17.381633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.490379 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:17.490338    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.618745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:17.618706    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:18.930130 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:18.930096    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:19.364028 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:19.363991    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:19.464077 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:19.464042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:20.001221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:20.001187    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:20.731430 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:20.731377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:20.831290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:20.831259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:21.321794 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:21.321756    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:21.563222 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:23:21.563181    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:23:21.887754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:21.887721    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.124411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:22.122047    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.365410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:22.359233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.486685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:22.486647    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.585772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:22.585716    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.916812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:22.916774    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.286303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.286266    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"]
Oct 30 21:23:23.287043 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287024    4852 cpu_manager.go:414] "containerMap: removing container" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287045    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287057    4852 cpu_manager.go:414] "containerMap: removing container" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287062    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287071    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287076    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287083    4852 cpu_manager.go:414] "containerMap: removing container" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287088    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287094    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287098    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287109    4852 cpu_manager.go:414] "containerMap: removing container" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy"
Oct 30 21:23:23.287111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287114    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287141    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287149    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287165    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287173    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287333    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="6db7f461-eb75-4c63-960a-ed412a088428" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287347    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287358    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287366    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287376    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0975483-3126-4e3a-8cf5-9eba54e64198" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287403    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="642feace-727f-49a7-8955-b895832d2540" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287417    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy"
Oct 30 21:23:23.287492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.287430    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd1dd97b-0870-4c4d-b313-685d43285e96" containerName="istio-proxy"
Oct 30 21:23:23.292621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.292603    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.302281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.302253    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-ffce13-openshift-default-dockercfg-qv6gg\""
Oct 30 21:23:23.305242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.305216    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"]
Oct 30 21:23:23.330493 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.330452    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.442553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.427601    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-envoy\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.467203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.467167    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-certs\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.467599 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.467574    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0685cd7c-3d01-4213-97e7-2215faee05ac-istiod-ca-cert\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.467820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.467802    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-socket\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.467987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.467970    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p46gt\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-kube-api-access-p46gt\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.473783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.473750    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-credential-socket\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.482475 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.474062    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-podinfo\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.496517 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.496483    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-data\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.496962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.496933    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-token\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.497277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.468421    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.600467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.599878    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0685cd7c-3d01-4213-97e7-2215faee05ac-istiod-ca-cert\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.600467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.599991    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-socket\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.600467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.600049    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p46gt\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-kube-api-access-p46gt\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.600467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.600120    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-credential-socket\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.600467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.600220    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-podinfo\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.600467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.600255    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-data\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.605604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.605537    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-token\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.605949 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.605923    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-envoy\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.606101 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.606079    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-certs\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.606801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.606778    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-certs\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.607636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.607611    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0685cd7c-3d01-4213-97e7-2215faee05ac-istiod-ca-cert\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.613415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.607982    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-socket\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.615375 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.615346    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-credential-socket\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.634484 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.630145    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.648278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.648240    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-envoy\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.649069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.649040    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-data\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.680536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.667005    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-token\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.680536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.680150    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-podinfo\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.694261 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.690131    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p46gt\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-kube-api-access-p46gt\") pod \"e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") " pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:23.914099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:23.914007    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:24.208256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.208228    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"]
Oct 30 21:23:24.208926 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:23:24.208903    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0685cd7c_3d01_4213_97e7_2215faee05ac.slice/crio-b9e7ad5b442cbd6d383c5abbf5b27d507897ac94cc5d6d01e0fc2d8bebcac338 WatchSource:0}: Error finding container b9e7ad5b442cbd6d383c5abbf5b27d507897ac94cc5d6d01e0fc2d8bebcac338: Status 404 returned error can't find the container with id b9e7ad5b442cbd6d383c5abbf5b27d507897ac94cc5d6d01e0fc2d8bebcac338
Oct 30 21:23:24.212082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.212047    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:23:24.212192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.212113    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:23:24.212192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.212142    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:23:24.253062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.252883    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:24.903466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.903424    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" event={"ID":"0685cd7c-3d01-4213-97e7-2215faee05ac","Type":"ContainerStarted","Data":"310f4bc2693e524c60cd5a2632615c9ee6a283e4dc96cea1c7d83272cf0228a6"}
Oct 30 21:23:24.903964 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.903513    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" event={"ID":"0685cd7c-3d01-4213-97e7-2215faee05ac","Type":"ContainerStarted","Data":"b9e7ad5b442cbd6d383c5abbf5b27d507897ac94cc5d6d01e0fc2d8bebcac338"}
Oct 30 21:23:24.914239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.914213    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:24.930103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:24.930052    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podStartSLOduration=1.930035524 podStartE2EDuration="1.930035524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:24.92475505 +0000 UTC m=+2713.039489242" watchObservedRunningTime="2026-10-30 21:23:24.930035524 +0000 UTC m=+2713.044769714"
Oct 30 21:23:25.007365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:25.007332    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:25.693381 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:25.693331    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:25.920952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:25.920924    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:26.630253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:26.630214    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:26.919581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:26.919490    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:26.920640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:26.920611    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:23:27.223843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:27.223764    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:27.320712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:27.320677    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:27.433131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:27.433096    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.204708 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:28.204669    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.302611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:28.302576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.398494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:28.398458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.128227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:29.128171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.210165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:29.210127    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.433469 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:29.411967    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:30.607891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:30.607856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:31.432703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:31.432672    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:32.227925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:32.227879    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:32.991750 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:32.991719    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:33.833590 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:33.833555    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:34.634485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:34.634449    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.326819 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:35.326785    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:36.050559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:36.050516    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:36.835378 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:36.835340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:37.617731 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:37.617672    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:38.351949 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:38.351912    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:40.280914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:40.280868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:40.366652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:40.366609    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:40.469724 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:40.469685    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:40.568705 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:40.568576    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:41.118352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.118305    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"]
Oct 30 21:23:41.123806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.123776    4852 cpu_manager.go:414] "containerMap: removing container" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy"
Oct 30 21:23:41.128499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.123969    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="618802d8-a3bc-4e09-846a-b3fd0ca1327c" containerName="istio-proxy"
Oct 30 21:23:41.128941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.128919    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c925dcc8-57c5-47aa-9dd4-a011fadf74da" containerName="istio-proxy"
Oct 30 21:23:41.165413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.157755    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.165413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.157635    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"]
Oct 30 21:23:41.165413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.162929    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-176db5-openshift-default-dockercfg-zm5vp\""
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.269874    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3683741c-abea-478c-bbfa-086679bda701-istiod-ca-cert\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.269930    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-socket\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.270599    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-istio-token\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.270822    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-data\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.270854    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3683741c-abea-478c-bbfa-086679bda701-istio-podinfo\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.270884    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-envoy\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.270991    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-certs\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.271059    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-credential-socket\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.272440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.271131    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d7l9\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-kube-api-access-8d7l9\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.376572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.376479    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-istio-token\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.377639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.377607    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-data\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.378248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.378228    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3683741c-abea-478c-bbfa-086679bda701-istio-podinfo\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.378863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.378843    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-envoy\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.379043 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.379025    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-certs\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.379211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.379193    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-credential-socket\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.379402 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.379366    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d7l9\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-kube-api-access-8d7l9\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.379567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.379548    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3683741c-abea-478c-bbfa-086679bda701-istiod-ca-cert\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.380263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.380242    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-socket\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.380606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.378177    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-data\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.382882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.379933    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-certs\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.383025 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.381652    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-credential-socket\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.383135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.382088    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-socket\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.383238 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.382825    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3683741c-abea-478c-bbfa-086679bda701-istiod-ca-cert\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.383600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.383577    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-envoy\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.511499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.503712    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3683741c-abea-478c-bbfa-086679bda701-istio-podinfo\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.511499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.506676    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-istio-token\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.525500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.519375    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d7l9\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-kube-api-access-8d7l9\") pod \"e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") " pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.548999 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.548706    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:41.875456 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:41.858179    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.252781 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:42.252738    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.392272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:42.392231    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:42.395779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:42.395749    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.647441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:42.644600    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:23:42.726541 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:42.726491    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.206450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.192220    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:43.206450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.192438    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"]
Oct 30 21:23:43.391866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.391747    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.393060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.393017    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.420495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.394072    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:23:43.467496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.456529    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.647667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.644462    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.647667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.645659    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:43.647667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.645987    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:23:43.696247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.696169    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.722572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.696925    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.841864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.841816    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.883494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.850540    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:23:43.883494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.851208    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:23:43.973093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.967524    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:23:43.973093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.971539    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:23:43.973093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:43.971718    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:23:44.210930 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.210881    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:44.354409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.348975    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:44.354409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.349413    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:44.474424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.470146    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 21:23:44.474424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.470523    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:23:44.498571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.498355    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" event={"ID":"3683741c-abea-478c-bbfa-086679bda701","Type":"ContainerStarted","Data":"1cf878db97300696755ba1f05f7aaba02106e2de8747479934f82be85c1ed3cf"}
Oct 30 21:23:44.500647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.500616    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:23:44.535670 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.529216    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:23:44.591408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.588748    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:23:44.610486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.598641    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:23:44.610486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.602941    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:23:44.835568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:44.835535    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:45.504602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:45.504567    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" event={"ID":"3683741c-abea-478c-bbfa-086679bda701","Type":"ContainerStarted","Data":"87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332"}
Oct 30 21:23:45.528006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:45.527956    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podStartSLOduration=4.527942404 podStartE2EDuration="4.527942404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:23:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:23:45.522675386 +0000 UTC m=+2733.637409576" watchObservedRunningTime="2026-10-30 21:23:45.527942404 +0000 UTC m=+2733.642676593"
Oct 30 21:23:45.550051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:45.550020    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:45.551988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:45.551959    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 21:23:45.552155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:45.552009    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 21:23:45.652302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:45.652268    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:46.412682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:46.412644    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:46.571414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:46.558645    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 21:23:46.571414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:46.559008    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 21:23:47.312821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:47.312783    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:47.554523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:47.554485    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 21:23:47.554831 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:47.554799    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 21:23:48.020161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:48.020121    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:48.574852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:48.574809    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 21:23:48.575023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:48.574876    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 21:23:48.826483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:48.826343    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.037065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:49.037032    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.170467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:49.164954    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.526622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:49.526588    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.564422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:49.562836    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 21:23:49.564422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:49.562923    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 21:23:49.728341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:49.728307    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.979793 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:49.979668    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:50.075322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:50.075265    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:50.553834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:50.553794    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 21:23:50.554161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:50.554129    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 21:23:50.790738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:50.790703    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:50.911630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:50.911534    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:50.994526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:50.994494    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:51.561411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:51.559815    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused" start-of-body=
Oct 30 21:23:51.561411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:51.559891    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.176:15021/healthz/ready\": dial tcp 10.133.0.176:15021: connect: connection refused"
Oct 30 21:23:51.589133 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:23:51.589041    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:23:51.640187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:51.640156    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:52.553065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:52.553027    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:52.555471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:52.555449    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:52.555860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:52.555839    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:52.556629 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:52.556609    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:23:53.510571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:53.510533    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:53.960337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:53.960259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:54.058715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:54.058675    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:54.410298 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:54.410266    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:55.150301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:55.150254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:55.260336 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:55.260287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:55.736929 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:55.736892    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:56.527418 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:56.527365    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:57.318699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:57.318661    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:58.021617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:58.021571    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:58.707885 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:58.707847    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:59.431979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:23:59.431927    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:00.116280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:00.116239    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:00.805817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:00.805784    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:01.631857 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:01.631820    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:02.187295 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.187251    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"]
Oct 30 21:24:02.187842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.187564    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj" podUID="83bce913-e437-4e71-8b9a-3a3ece1096c0" containerName="maas-api" containerID="cri-o://58247c6b6fe93f8a6163521b37dd1809dde1d8ef30c8359d31a9e395e30b6e4d" gracePeriod=30
Oct 30 21:24:02.245651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.245607    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:02.423787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.423680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:02.661120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.658226    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:02.777532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.777495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:02.778113 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.778084    4852 generic.go:356] "Generic (PLEG): container finished" podID="83bce913-e437-4e71-8b9a-3a3ece1096c0" containerID="58247c6b6fe93f8a6163521b37dd1809dde1d8ef30c8359d31a9e395e30b6e4d" exitCode=0
Oct 30 21:24:02.778352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.778336    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj" event={"ID":"83bce913-e437-4e71-8b9a-3a3ece1096c0","Type":"ContainerDied","Data":"58247c6b6fe93f8a6163521b37dd1809dde1d8ef30c8359d31a9e395e30b6e4d"}
Oct 30 21:24:02.805043 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.805012    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:24:02.870257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.870216    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83bce913-e437-4e71-8b9a-3a3ece1096c0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83bce913-e437-4e71-8b9a-3a3ece1096c0-openshift-service-ca\") pod \"83bce913-e437-4e71-8b9a-3a3ece1096c0\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") "
Oct 30 21:24:02.870482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.870315    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83bce913-e437-4e71-8b9a-3a3ece1096c0-kube-api-access-tjfgb\" (UniqueName: \"kubernetes.io/projected/83bce913-e437-4e71-8b9a-3a3ece1096c0-kube-api-access-tjfgb\") pod \"83bce913-e437-4e71-8b9a-3a3ece1096c0\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") "
Oct 30 21:24:02.870482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.870435    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-maas-api-tls\") pod \"83bce913-e437-4e71-8b9a-3a3ece1096c0\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") "
Oct 30 21:24:02.871745 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.871697    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83bce913-e437-4e71-8b9a-3a3ece1096c0-openshift-service-ca" pod "83bce913-e437-4e71-8b9a-3a3ece1096c0" (UID: "83bce913-e437-4e71-8b9a-3a3ece1096c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:02.873790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.873759    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83bce913-e437-4e71-8b9a-3a3ece1096c0-kube-api-access-tjfgb" pod "83bce913-e437-4e71-8b9a-3a3ece1096c0" (UID: "83bce913-e437-4e71-8b9a-3a3ece1096c0"). InnerVolumeSpecName "kube-api-access-tjfgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:02.873930 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.873913    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-maas-api-tls" pod "83bce913-e437-4e71-8b9a-3a3ece1096c0" (UID: "83bce913-e437-4e71-8b9a-3a3ece1096c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:02.972119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.972021    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-metrics-certs\") pod \"83bce913-e437-4e71-8b9a-3a3ece1096c0\" (UID: \"83bce913-e437-4e71-8b9a-3a3ece1096c0\") "
Oct 30 21:24:02.972809 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.972783    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83bce913-e437-4e71-8b9a-3a3ece1096c0-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:02.972809 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.972809    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjfgb\" (UniqueName: \"kubernetes.io/projected/83bce913-e437-4e71-8b9a-3a3ece1096c0-kube-api-access-tjfgb\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:02.972982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.972825    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:02.974324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:02.974300    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-metrics-certs" pod "83bce913-e437-4e71-8b9a-3a3ece1096c0" (UID: "83bce913-e437-4e71-8b9a-3a3ece1096c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:03.075044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.074992    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83bce913-e437-4e71-8b9a-3a3ece1096c0-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:03.179820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.179786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:03.551206 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.551170    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:03.788237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.788208    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"
Oct 30 21:24:03.788438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.788246    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj" event={"ID":"83bce913-e437-4e71-8b9a-3a3ece1096c0","Type":"ContainerDied","Data":"dd743d4c01c2316747b6df8a9920f90e11309c3d0b4189dd3b4684b40fa3239d"}
Oct 30 21:24:03.788438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.788308    4852 scope.go:122] "RemoveContainer" containerID="58247c6b6fe93f8a6163521b37dd1809dde1d8ef30c8359d31a9e395e30b6e4d"
Oct 30 21:24:03.816844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.816760    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"]
Oct 30 21:24:03.821306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:03.821266    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-345bc4-688689447c-rdswj"]
Oct 30 21:24:04.006341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:04.006305    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83bce913-e437-4e71-8b9a-3a3ece1096c0" path="/var/lib/kubelet/pods/83bce913-e437-4e71-8b9a-3a3ece1096c0/volumes"
Oct 30 21:24:04.480337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:04.480301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:05.252310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:05.252277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:05.416723 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:05.416693    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:06.296545 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:06.296513    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.042347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:07.042303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.501928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:07.501833    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.818857 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:07.818817    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:08.276899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:08.275739    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:08.716022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:08.715826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:08.804712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:08.804677    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:09.097891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.097855    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"]
Oct 30 21:24:09.099050 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.099027    4852 cpu_manager.go:414] "containerMap: removing container" podUID="83bce913-e437-4e71-8b9a-3a3ece1096c0" containerName="maas-api"
Oct 30 21:24:09.099121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.099055    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83bce913-e437-4e71-8b9a-3a3ece1096c0" containerName="maas-api"
Oct 30 21:24:09.099326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.099310    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="83bce913-e437-4e71-8b9a-3a3ece1096c0" containerName="maas-api"
Oct 30 21:24:09.105371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.105337    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.110183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.110159    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5506f4-openshift-default-dockercfg-jjrlt\""
Oct 30 21:24:09.111366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.111341    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"]
Oct 30 21:24:09.136119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.136074    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214122    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlnlh\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-kube-api-access-xlnlh\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214366    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-token\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214494    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-data\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214619    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-socket\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214657    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istiod-ca-cert\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214684    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-credential-socket\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214717    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-podinfo\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214747    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-certs\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.219444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.214783    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-envoy\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.316789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.316748    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-token\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.323567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.323534    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-data\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.324423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.324373    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-socket\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.324936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.324919    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istiod-ca-cert\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.327479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.327456    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-credential-socket\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.347850    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-podinfo\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.347895    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-certs\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.347932    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-envoy\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.348060    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlnlh\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-kube-api-access-xlnlh\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.317641    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:09.349461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.324862    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-socket\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.347763    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-credential-socket\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.349534    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-certs\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.324194    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-data\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.349962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.325639    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istiod-ca-cert\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.365063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.364449    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-token\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.369777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.369716    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-envoy\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.382406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.374853    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-podinfo\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.382406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.378036    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlnlh\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-kube-api-access-xlnlh\") pod \"e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") " pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.423686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.423653    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:09.714979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.714897    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:09.810773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.810740    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"]
Oct 30 21:24:09.811180 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:24:09.811068    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b136f0_4b7e_4a66_bb5f_e6f296a7c19d.slice/crio-c196bbe6e8c8b25d385c483928903abd5f97d3b9ff2cf85c595d91f84a7c6104 WatchSource:0}: Error finding container c196bbe6e8c8b25d385c483928903abd5f97d3b9ff2cf85c595d91f84a7c6104: Status 404 returned error can't find the container with id c196bbe6e8c8b25d385c483928903abd5f97d3b9ff2cf85c595d91f84a7c6104
Oct 30 21:24:09.814168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.814138    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:09.814271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.814210    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:09.814271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.814247    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:09.962191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:09.962158    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" event={"ID":"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d","Type":"ContainerStarted","Data":"c196bbe6e8c8b25d385c483928903abd5f97d3b9ff2cf85c595d91f84a7c6104"}
Oct 30 21:24:10.060189 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.058570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.153296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.153250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.382038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.382005    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"]
Oct 30 21:24:10.382251 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.382228    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy" containerID="cri-o://ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5" gracePeriod=30
Oct 30 21:24:10.398069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.398020    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.414179 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.414141    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"]
Oct 30 21:24:10.414857 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.414803    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy" containerID="cri-o://c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770" gracePeriod=30
Oct 30 21:24:10.886761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.886726    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.990762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:10.990713    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" event={"ID":"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d","Type":"ContainerStarted","Data":"d67730157f6a28cedacf449392389f300e00ed697f042edbaa6864e2336df756"}
Oct 30 21:24:11.018439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:11.018372    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" podStartSLOduration=2.018356419 podStartE2EDuration="2.018356419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:11.01412079 +0000 UTC m=+2759.128854979" watchObservedRunningTime="2026-10-30 21:24:11.018356419 +0000 UTC m=+2759.133090603"
Oct 30 21:24:11.425114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:11.425079    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:11.432143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:11.432103    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:11.937852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:11.937811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:12.038660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:12.038617    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:12.039941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:12.039910    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:24:12.828795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:12.828758    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.289235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:13.289202    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.445832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:13.445798    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.610905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:13.610811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.729868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:13.729834    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:24:13.730095 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:13.729888    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:24:13.781866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:13.781825    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:24:13.782076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:13.781890    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:24:14.118603 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:14.118561    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:14.698971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:14.698881    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:14.795653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:14.795616    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:15.424298 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.424267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:15.758294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.758264    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:24:15.796729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.796689    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:24:15.857057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857023    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-socket\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.857263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857136    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-kube-api-access-rd2s7\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-kube-api-access-rd2s7\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.857263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857190    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-podinfo\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.857421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857266    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istiod-ca-cert\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.857421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857299    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-data\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.857421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857360    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-socket" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.857421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857379    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-credential-socket\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.857647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857537    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-envoy\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.857701 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857641    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-data" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.857701 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857680    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-credential-socket" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.857816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.857742    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istiod-ca-cert" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:15.858155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.858132    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-certs\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.858261 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.858186    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-token\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-token\") pod \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\" (UID: \"e3c32cf1-82b7-47b4-b01e-91f1d80192ac\") "
Oct 30 21:24:15.858496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.858469    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-certs" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.859129 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859100    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.859239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859135    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.859239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859150    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.859239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859166    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.859239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859181    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.859833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859807    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-envoy" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.859907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859861    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-podinfo" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:24:15.860007 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.859984    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-kube-api-access-rd2s7" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "kube-api-access-rd2s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:15.860633 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.860616    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-token" pod "e3c32cf1-82b7-47b4-b01e-91f1d80192ac" (UID: "e3c32cf1-82b7-47b4-b01e-91f1d80192ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:15.960294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960216    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-certs\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960258    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-socket\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960361    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-kube-api-access-sq22s\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-kube-api-access-sq22s\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960427    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-data\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960496    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-credential-socket\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960546    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-certs" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.960698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960614    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-istio-token\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-istio-token\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960657    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-socket" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.960698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960671    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db966d6e-90cc-450c-9842-18807aac3c9d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db966d6e-90cc-450c-9842-18807aac3c9d-istio-podinfo\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960710    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-envoy\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960749    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-data" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.960911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960833    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db966d6e-90cc-450c-9842-18807aac3c9d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db966d6e-90cc-450c-9842-18807aac3c9d-istiod-ca-cert\") pod \"db966d6e-90cc-450c-9842-18807aac3c9d\" (UID: \"db966d6e-90cc-450c-9842-18807aac3c9d\") "
Oct 30 21:24:15.960911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.960850    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-credential-socket" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.961274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961223    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db966d6e-90cc-450c-9842-18807aac3c9d-istiod-ca-cert" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:15.961577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961559    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd2s7\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-kube-api-access-rd2s7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961580    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961590    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961602    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961611    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961619    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db966d6e-90cc-450c-9842-18807aac3c9d-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961626    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3c32cf1-82b7-47b4-b01e-91f1d80192ac-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961634    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.961863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.961642    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:15.963281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.963256    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-envoy" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:15.963368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.963320    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db966d6e-90cc-450c-9842-18807aac3c9d-istio-podinfo" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:24:15.963368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.963354    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-kube-api-access-sq22s" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "kube-api-access-sq22s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:15.963525 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:15.963382    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-istio-token" pod "db966d6e-90cc-450c-9842-18807aac3c9d" (UID: "db966d6e-90cc-450c-9842-18807aac3c9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:16.063951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.063910    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:16.063951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.063947    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db966d6e-90cc-450c-9842-18807aac3c9d-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:16.063951 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.063960    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db966d6e-90cc-450c-9842-18807aac3c9d-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:16.064212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.063969    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq22s\" (UniqueName: \"kubernetes.io/projected/db966d6e-90cc-450c-9842-18807aac3c9d-kube-api-access-sq22s\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:16.082251 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.082218    4852 generic.go:356] "Generic (PLEG): container finished" podID="db966d6e-90cc-450c-9842-18807aac3c9d" containerID="c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770" exitCode=0
Oct 30 21:24:16.082470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.082267    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"
Oct 30 21:24:16.082470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.082303    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" event={"ID":"db966d6e-90cc-450c-9842-18807aac3c9d","Type":"ContainerDied","Data":"c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770"}
Oct 30 21:24:16.082470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.082365    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp" event={"ID":"db966d6e-90cc-450c-9842-18807aac3c9d","Type":"ContainerDied","Data":"6ebc1df0ca861ffd52c28787821bd39a838214b01f0e908336a5db20adc94012"}
Oct 30 21:24:16.082470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.082423    4852 scope.go:122] "RemoveContainer" containerID="c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770"
Oct 30 21:24:16.084228 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.084207    4852 generic.go:356] "Generic (PLEG): container finished" podID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerID="ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5" exitCode=0
Oct 30 21:24:16.084365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.084251    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"
Oct 30 21:24:16.084365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.084285    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" event={"ID":"e3c32cf1-82b7-47b4-b01e-91f1d80192ac","Type":"ContainerDied","Data":"ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5"}
Oct 30 21:24:16.084365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.084330    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m" event={"ID":"e3c32cf1-82b7-47b4-b01e-91f1d80192ac","Type":"ContainerDied","Data":"28d9e874ecce35682ba522922a22771f90e30a9cd0acfd13bc788f472ba7cf4e"}
Oct 30 21:24:16.096164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.096142    4852 scope.go:122] "RemoveContainer" containerID="c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770"
Oct 30 21:24:16.096521 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:24:16.096499    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770\": container with ID starting with c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770 not found: ID does not exist" containerID="c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770"
Oct 30 21:24:16.096588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.096529    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770"} err="failed to get container status \"c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770\": rpc error: code = NotFound desc = could not find container \"c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770\": container with ID starting with c39091bfa4cdee90a1122565570ceac16bd251a47e5ed148d9616768dd4e8770 not found: ID does not exist"
Oct 30 21:24:16.096588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.096548    4852 scope.go:122] "RemoveContainer" containerID="ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5"
Oct 30 21:24:16.108228 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.108207    4852 scope.go:122] "RemoveContainer" containerID="ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5"
Oct 30 21:24:16.108618 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:24:16.108592    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5\": container with ID starting with ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5 not found: ID does not exist" containerID="ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5"
Oct 30 21:24:16.108795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.108626    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5"} err="failed to get container status \"ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5\": rpc error: code = NotFound desc = could not find container \"ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5\": container with ID starting with ec7dde1a5e889f487784f6fc53e8eb165ccbf9dd7cc972aafb2fe69a4e0e79b5 not found: ID does not exist"
Oct 30 21:24:16.109962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.109939    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"]
Oct 30 21:24:16.115216 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.115191    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-9hc4m"]
Oct 30 21:24:16.127966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.127939    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"]
Oct 30 21:24:16.132276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.132251    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-345bc4-openshift-default-c59dc5c89-mrttp"]
Oct 30 21:24:16.402626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.402590    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"]
Oct 30 21:24:16.403621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.403582    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy"
Oct 30 21:24:16.403621 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.403610    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy"
Oct 30 21:24:16.403794 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.403656    4852 cpu_manager.go:414] "containerMap: removing container" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy"
Oct 30 21:24:16.403794 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.403670    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy"
Oct 30 21:24:16.403901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.403885    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" containerName="istio-proxy"
Oct 30 21:24:16.403938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.403917    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" containerName="istio-proxy"
Oct 30 21:24:16.410176 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.410152    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.417068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.416607    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2ac526-serving-cert\""
Oct 30 21:24:16.420566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.420541    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"]
Oct 30 21:24:16.462070 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.462033    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:16.563727 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.563666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:16.576367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.576337    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-metrics-certs\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.576515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.576458    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d9cd02-b0eb-4e2a-a614-353161d08efc-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.576515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.576488    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.576758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.576736    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j285\" (UniqueName: \"kubernetes.io/projected/e2d9cd02-b0eb-4e2a-a614-353161d08efc-kube-api-access-2j285\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.671197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.671100    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:16.685701 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.682843    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-metrics-certs\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.686142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.686112    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d9cd02-b0eb-4e2a-a614-353161d08efc-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.686346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.686320    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.686675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.686643    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j285\" (UniqueName: \"kubernetes.io/projected/e2d9cd02-b0eb-4e2a-a614-353161d08efc-kube-api-access-2j285\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.688881 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.688118    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d9cd02-b0eb-4e2a-a614-353161d08efc-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.701168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.697325    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-metrics-certs\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.704884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.704838    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-maas-api-tls\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.705124 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.705088    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j285\" (UniqueName: \"kubernetes.io/projected/e2d9cd02-b0eb-4e2a-a614-353161d08efc-kube-api-access-2j285\") pod \"maas-api-e2e-inf-a-2ac526-7647f45775-sr85t\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:16.753239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:16.753209    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:17.358309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:17.358271    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"]
Oct 30 21:24:17.365763 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:17.365737    4852 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:24:17.515639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:17.515603    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:17.752452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:17.752427    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:18.006548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:18.006522    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db966d6e-90cc-450c-9842-18807aac3c9d" path="/var/lib/kubelet/pods/db966d6e-90cc-450c-9842-18807aac3c9d/volumes"
Oct 30 21:24:18.007101 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:18.007086    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3c32cf1-82b7-47b4-b01e-91f1d80192ac" path="/var/lib/kubelet/pods/e3c32cf1-82b7-47b4-b01e-91f1d80192ac/volumes"
Oct 30 21:24:18.108467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:18.108334    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t" event={"ID":"e2d9cd02-b0eb-4e2a-a614-353161d08efc","Type":"ContainerStarted","Data":"cabab92c5416b0774fcd203437f722acf371463e75e5acb284c64f3454d7ace3"}
Oct 30 21:24:18.108467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:18.108456    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t" event={"ID":"e2d9cd02-b0eb-4e2a-a614-353161d08efc","Type":"ContainerStarted","Data":"e2abe7ab870b744e2c08cf54e380e677cd1a0e8be96abe2c3952803dd4504297"}
Oct 30 21:24:18.108694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:18.108493    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:18.225578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:18.225548    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:18.690810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:18.690748    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:19.051410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.045846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:19.209466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.068335    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t" podStartSLOduration=2.686770906 podStartE2EDuration="3.068310199s" totalImagesPullingTime="381.539293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:18.136688695 +0000 UTC m=+2766.251422886" watchObservedRunningTime="2026-10-30 21:24:19.068310199 +0000 UTC m=+2767.183044400"
Oct 30 21:24:19.209466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.069949    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"]
Oct 30 21:24:19.308897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.307269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:19.414420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.392876    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.419992 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.419256    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-9b78e4-openshift-default-dockercfg-2r26q\""
Oct 30 21:24:19.479418 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.474051    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"]
Oct 30 21:24:19.509507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.505901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.613934    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-envoy\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.614017    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwg48\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-kube-api-access-rwg48\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.614277    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-token\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.614362    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-socket\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.614578    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-podinfo\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.614800    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-credential-socket\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.614847    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-certs\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.614891    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istiod-ca-cert\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.623466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.615022    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-data\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735175    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-token\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735245    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-socket\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735337    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-podinfo\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735448    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-credential-socket\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735478    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-certs\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735508    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istiod-ca-cert\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735564    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-data\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735644    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-envoy\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.735681    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwg48\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-kube-api-access-rwg48\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.737410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.736667    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-socket\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.766414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.765777    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istiod-ca-cert\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.766414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.766067    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-credential-socket\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.766414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.766316    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-certs\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.766777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.766565    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-data\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.799506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.799436    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwg48\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-kube-api-access-rwg48\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.837060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.811375    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-envoy\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.837060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.817994    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-podinfo\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:19.883470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:19.874004    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-token\") pod \"e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") " pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:20.058755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:20.058712    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:20.288412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:20.284470    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:20.441485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:20.439887    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:20.655409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:20.655230    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:20.688577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:20.683500    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:20.907658 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:20.895482    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 21:24:20.907658 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:20.895896    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.142198 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.142109    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:21.425415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.417465    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.473610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.473117    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"]
Oct 30 21:24:21.479488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.476741    4852 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-77f556d647-xr9v5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:24:21.479488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.476805    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:21.541448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.514598    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:21.541448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.514731    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:21.541448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.514782    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:21.574465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.545511    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.574465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.552587    4852 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/health\": context deadline exceeded"
Oct 30 21:24:21.684569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.652886    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.684569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.675509    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:21.684569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.683303    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:21.700240 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.684476    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.700240 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.695043    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.744575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.708158    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.744575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.708438    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.744575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.708645    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:24:21.744575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.708698    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:24:21.744575 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:24:21.727123    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d9690d9_dfb2_47c7_805e_0a84c8a89b42.slice/crio-5d993eda9c2e67e388aca3f37c4ead7bd5a4ea6dd6ebf3627709a8c3a16094ec WatchSource:0}: Error finding container 5d993eda9c2e67e388aca3f37c4ead7bd5a4ea6dd6ebf3627709a8c3a16094ec: Status 404 returned error can't find the container with id 5d993eda9c2e67e388aca3f37c4ead7bd5a4ea6dd6ebf3627709a8c3a16094ec
Oct 30 21:24:21.955642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.955345    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5" podUID="06b775dd-29f6-409e-8725-a476c704e31c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.958802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.957479    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 21:24:21.962483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.959595    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:24:21.962483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:21.959917    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:24:22.148301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.148250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:22.149516 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:24:22.149483    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:24:22.519417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.420824    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:22.566536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.551884    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:22.613417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.605478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:22.694422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.680912    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:24:22.697904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.697866    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:22.698466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.698435    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:22.719916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.719874    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:22.720554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.720506    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.86:8000/ready\": context deadline exceeded"
Oct 30 21:24:22.763135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.763075    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:22.888280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.888184    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5"
Oct 30 21:24:22.888594 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.888574    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:24:22.888755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.888743    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:24:22.888871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.888852    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" event={"ID":"5d9690d9-dfb2-47c7-805e-0a84c8a89b42","Type":"ContainerStarted","Data":"5d993eda9c2e67e388aca3f37c4ead7bd5a4ea6dd6ebf3627709a8c3a16094ec"}
Oct 30 21:24:22.889013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.889000    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:24:22.889137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.889125    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:24:22.889291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.889279    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:24:22.889446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:22.889432    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:24:23.094631 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.067093    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:24:23.118082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.111276    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:24:23.118082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.109369    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:23.118082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.111329    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:24:23.118082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.111480    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:24:23.118082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.111517    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:24:23.118082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.111562    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:24:23.118082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.112253    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:24:23.686377 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.686343    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" event={"ID":"5d9690d9-dfb2-47c7-805e-0a84c8a89b42","Type":"ContainerStarted","Data":"437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af"}
Oct 30 21:24:23.705605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:23.705547    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podStartSLOduration=5.705529882 podStartE2EDuration="5.705529882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:23.704650039 +0000 UTC m=+2771.819384229" watchObservedRunningTime="2026-10-30 21:24:23.705529882 +0000 UTC m=+2771.820264072"
Oct 30 21:24:24.072267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:24.069853    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:24.079594 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:24.079559    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:24.081271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:24.081242    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:24.424853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:24.424775    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:24:24.697041 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:24.696942    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:24.698115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:24.698085    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:24:24.821326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:24.821292    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:25.838653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:25.838615    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:26.003470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:26.003432    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:26.108330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:26.108250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:26.724756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:26.724719    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:27.238779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:27.238732    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:27.339915 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:27.339863    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:27.617410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:27.616710    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:27.960619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:27.960525    4852 patch_prober.go:29] interesting pod/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:24:27.960816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:27.960629    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:28.442457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:28.442406    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.230447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:29.230372    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.928864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:29.928822    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:30.641217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:30.641180    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:30.786566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:30.786536    4852 generic.go:356] "Generic (PLEG): container finished" podID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerID="310f4bc2693e524c60cd5a2632615c9ee6a283e4dc96cea1c7d83272cf0228a6" exitCode=0
Oct 30 21:24:30.786753 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:30.786611    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" event={"ID":"0685cd7c-3d01-4213-97e7-2215faee05ac","Type":"ContainerDied","Data":"310f4bc2693e524c60cd5a2632615c9ee6a283e4dc96cea1c7d83272cf0228a6"}
Oct 30 21:24:30.787120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:30.787105    4852 scope.go:122] "RemoveContainer" containerID="310f4bc2693e524c60cd5a2632615c9ee6a283e4dc96cea1c7d83272cf0228a6"
Oct 30 21:24:30.789282 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:30.789256    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:30.789438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:30.789322    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:30.789438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:30.789355    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:24:31.532834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:31.532790    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:31.797228 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:31.797146    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" event={"ID":"0685cd7c-3d01-4213-97e7-2215faee05ac","Type":"ContainerStarted","Data":"307698abde47f9535848da222d9c06005cbdb24f1c51ca3d6650c70f2f63b48e"}
Oct 30 21:24:31.914360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:31.914331    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:24:31.914563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:31.914511    4852 patch_prober.go:29] interesting pod/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 21:24:31.914613 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:31.914558    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 21:24:32.281371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:32.281332    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:32.915092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:32.915057    4852 patch_prober.go:29] interesting pod/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 21:24:32.915533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:32.915113    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 21:24:32.998463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:32.998428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:33.847870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:33.847836    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:33.915325 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:33.915270    4852 patch_prober.go:29] interesting pod/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 21:24:33.916040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:33.915989    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 21:24:34.733784 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:34.733726    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:34.914790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:34.914752    4852 patch_prober.go:29] interesting pod/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 21:24:34.915032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:34.914815    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 21:24:35.629119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:35.629064    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:35.915159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:35.915071    4852 patch_prober.go:29] interesting pod/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 21:24:35.915159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:35.915130    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 21:24:36.320831 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:36.320795    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.496920 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:36.496881    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.604972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:36.604883    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.910278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:36.910185    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.914526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:36.914499    4852 patch_prober.go:29] interesting pod/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused" start-of-body=
Oct 30 21:24:36.914627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:36.914571    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.175:15021/healthz/ready\": dial tcp 10.133.0.175:15021: connect: connection refused"
Oct 30 21:24:37.035253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:37.035215    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:37.175273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:37.175178    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:37.265743 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:37.265711    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:37.919757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:37.919724    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:24:37.920257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:37.920238    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:24:37.920937 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:37.920917    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:24:37.925287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:37.925262    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:38.841851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:38.841819    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:39.605744 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:39.605710    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:40.450750 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:40.450713    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:41.415338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:41.415306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:42.236615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:42.236578    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:43.000382 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:43.000347    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:43.832319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:43.832285    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:43.935251 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:43.935146    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:44.197868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:44.197773    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"]
Oct 30 21:24:44.198498 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:44.198012    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy" containerID="cri-o://13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5" gracePeriod=30
Oct 30 21:24:44.218730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:44.218697    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:44.883823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:44.883788    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:44.992867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:44.991977    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:45.338236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:45.338171    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:45.812563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:45.812506    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:45.912698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:45.912657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.031873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:46.031835    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.880228 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:46.880181    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.631714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:47.631673    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.706447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:47.706354    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.815463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:47.815427    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:48.695089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:48.695058    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:49.433496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.433463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:49.505487 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.505452    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:24:49.526711 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.526669    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-istio-token\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-istio-token\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.528353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.528325    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-envoy\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.528587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.528568    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-credential-socket\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.528752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.528726    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-kube-api-access-9wqt8\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-kube-api-access-9wqt8\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.528978 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.528773    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-certs\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.528978 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.528823    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-data\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.529259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.528981    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5664b217-4b0f-434a-8606-123811d27c4b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5664b217-4b0f-434a-8606-123811d27c4b-istiod-ca-cert\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.529259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.529012    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-socket\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.529259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.529129    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5664b217-4b0f-434a-8606-123811d27c4b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5664b217-4b0f-434a-8606-123811d27c4b-istio-podinfo\") pod \"5664b217-4b0f-434a-8606-123811d27c4b\" (UID: \"5664b217-4b0f-434a-8606-123811d27c4b\") "
Oct 30 21:24:49.530538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.529711    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-certs" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:49.530538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.530014    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-data" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:49.530538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.530122    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.530538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.530258    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-socket" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:49.530538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.530273    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-credential-socket" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:49.530538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.530438    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5664b217-4b0f-434a-8606-123811d27c4b-istiod-ca-cert" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:49.532138 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.532093    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-istio-token" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:49.532940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.532903    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-kube-api-access-9wqt8" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "kube-api-access-9wqt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:49.533035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.532959    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:49.533208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.533184    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5664b217-4b0f-434a-8606-123811d27c4b-istio-podinfo" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:24:49.535445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.535421    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-envoy" pod "5664b217-4b0f-434a-8606-123811d27c4b" (UID: "5664b217-4b0f-434a-8606-123811d27c4b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:24:49.631586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631550    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5664b217-4b0f-434a-8606-123811d27c4b-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.631586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631583    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.631586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631593    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.631914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631604    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.631914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631614    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wqt8\" (UniqueName: \"kubernetes.io/projected/5664b217-4b0f-434a-8606-123811d27c4b-kube-api-access-9wqt8\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.631914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631623    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.631914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631633    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5664b217-4b0f-434a-8606-123811d27c4b-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.631914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.631643    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5664b217-4b0f-434a-8606-123811d27c4b-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:49.814628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:49.814574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:50.154898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.154797    4852 generic.go:356] "Generic (PLEG): container finished" podID="5664b217-4b0f-434a-8606-123811d27c4b" containerID="13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5" exitCode=0
Oct 30 21:24:50.155091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.154985    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" event={"ID":"5664b217-4b0f-434a-8606-123811d27c4b","Type":"ContainerDied","Data":"13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5"}
Oct 30 21:24:50.155091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.155069    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr" event={"ID":"5664b217-4b0f-434a-8606-123811d27c4b","Type":"ContainerDied","Data":"873d232cf86a0c5c19e8183e7cb5a51727c84048a572addad470859147261ec7"}
Oct 30 21:24:50.155208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.155118    4852 scope.go:122] "RemoveContainer" containerID="13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5"
Oct 30 21:24:50.155523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.155329    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"
Oct 30 21:24:50.168553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.168530    4852 scope.go:122] "RemoveContainer" containerID="13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5"
Oct 30 21:24:50.168836 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:24:50.168810    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5\": container with ID starting with 13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5 not found: ID does not exist" containerID="13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5"
Oct 30 21:24:50.168957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.168845    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5"} err="failed to get container status \"13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5\": rpc error: code = NotFound desc = could not find container \"13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5\": container with ID starting with 13a6d1229d7468a923296d81a0b3e711d592d15498c29d1f3d8a271db6f9d7a5 not found: ID does not exist"
Oct 30 21:24:50.181273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.181236    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"]
Oct 30 21:24:50.185653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.185627    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-a574c2-openshift-default-5c5f754885-x56cr"]
Oct 30 21:24:50.769636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.769600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:50.937771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:50.937706    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.541505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:51.541473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.560478 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:24:51.560444    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:24:52.008153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:52.008124    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5664b217-4b0f-434a-8606-123811d27c4b" path="/var/lib/kubelet/pods/5664b217-4b0f-434a-8606-123811d27c4b/volumes"
Oct 30 21:24:52.327677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:52.327580    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:53.091210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:53.091176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:54.119089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:54.119033    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:54.889311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:54.889267    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"]
Oct 30 21:24:54.904375 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:54.904335    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"]
Oct 30 21:24:54.916476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:54.916444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:55.019032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:55.018996    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:55.784364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:55.784328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:56.575154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:56.575073    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:57.312081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:57.312043    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:58.016438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:58.016358    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:58.726753 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:58.726712    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:59.500358 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:59.500310    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:59.890524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:59.890438    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" podUID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" containerName="payload-processing" containerID="cri-o://735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d" gracePeriod=25
Oct 30 21:24:59.905165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:24:59.905124    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" podUID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" containerName="payload-pre-processing" containerID="cri-o://fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324" gracePeriod=25
Oct 30 21:25:00.223840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:00.223747    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:01.144436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:01.144398    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:01.988337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:01.988290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:02.688544 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:02.688509    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:02.736764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:02.736711    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" podUID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:25:02.736764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:02.736755    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" podUID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:25:03.520617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:03.520575    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:04.158706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.158664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:04.327820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.327779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:04.480776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.480671    4852 generic.go:356] "Generic (PLEG): container finished" podID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerID="d67730157f6a28cedacf449392389f300e00ed697f042edbaa6864e2336df756" exitCode=0
Oct 30 21:25:04.480776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.480739    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" event={"ID":"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d","Type":"ContainerDied","Data":"d67730157f6a28cedacf449392389f300e00ed697f042edbaa6864e2336df756"}
Oct 30 21:25:04.481426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.481356    4852 scope.go:122] "RemoveContainer" containerID="d67730157f6a28cedacf449392389f300e00ed697f042edbaa6864e2336df756"
Oct 30 21:25:04.484351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.484309    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:25:04.484487 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.484465    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:25:04.484606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:04.484579    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:25:05.254410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:05.252671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:05.354419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:05.352472    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:05.495011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:05.494971    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" event={"ID":"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d","Type":"ContainerStarted","Data":"0fd184959dfd1fbf24abaa5c129ed4d07240d2befb51c3efb4bc4043f11e729c"}
Oct 30 21:25:05.780284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:05.780244    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:06.199376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:06.199286    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:06.424863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:06.424826    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:25:06.425295 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:06.425041    4852 patch_prober.go:29] interesting pod/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:25:06.425295 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:06.425086    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 21:25:06.943292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:06.943249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:07.425169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:07.425130    4852 patch_prober.go:29] interesting pod/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:25:07.425607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:07.425195    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 21:25:07.741109 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:07.740977    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:08.424540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:08.424504    4852 patch_prober.go:29] interesting pod/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:25:08.424737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:08.424562    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 21:25:08.480453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:08.480412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:09.298350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:09.298308    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:09.431226 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:09.428626    4852 patch_prober.go:29] interesting pod/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:25:09.431226 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:09.428686    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 21:25:10.013623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.013566    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:10.425253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.425157    4852 patch_prober.go:29] interesting pod/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused" start-of-body=
Oct 30 21:25:10.425448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.425246    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.177:15021/healthz/ready\": dial tcp 10.133.0.177:15021: connect: connection refused"
Oct 30 21:25:10.720555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.720456    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"]
Oct 30 21:25:10.721966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.721933    4852 cpu_manager.go:414] "containerMap: removing container" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy"
Oct 30 21:25:10.722158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.722142    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy"
Oct 30 21:25:10.722588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.722568    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="5664b217-4b0f-434a-8606-123811d27c4b" containerName="istio-proxy"
Oct 30 21:25:10.756215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.756176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:10.777702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.777665    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.787693 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.787660    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a3fb70-serving-cert\""
Oct 30 21:25:10.790303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.790270    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-metrics-certs\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.790559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.790538    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.790783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.790763    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.790918 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.790903    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfhb2\" (UniqueName: \"kubernetes.io/projected/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-kube-api-access-tfhb2\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.795882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.795846    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"]
Oct 30 21:25:10.929812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.913599    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-metrics-certs\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.929812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.913681    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.929812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.913803    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:10.929812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:10.913851    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhb2\" (UniqueName: \"kubernetes.io/projected/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-kube-api-access-tfhb2\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:11.051950 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:25:11.048528    4852 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-serving-cert: secret "maas-api-e2e-inf-b-a3fb70-serving-cert" not found
Oct 30 21:25:11.051950 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:25:11.048647    4852 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls podName:7894ad8b-11d2-480f-92f1-8d18a3c2ba62 nodeName:}" failed. No retries permitted until 2026-10-30 21:25:11.548619225 +0000 UTC m=+2819.663353414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls") pod "maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv" (UID: "7894ad8b-11d2-480f-92f1-8d18a3c2ba62") : secret "maas-api-e2e-inf-b-a3fb70-serving-cert" not found
Oct 30 21:25:11.089419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.067770    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:11.089419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.070991    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:11.089419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.083140    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhb2\" (UniqueName: \"kubernetes.io/projected/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-kube-api-access-tfhb2\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:11.143844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.143470    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-metrics-certs\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:11.188717 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.188685    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:11.430452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.430358    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:25:11.430917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.430896    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:25:11.431718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.431702    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:25:11.585578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.585533    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:11.587993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.587963    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls\") pod \"maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:11.710755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.710639    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:11.748904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:11.748877    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:12.177193 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:12.177161    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"]
Oct 30 21:25:12.177193 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:25:12.177179    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7894ad8b_11d2_480f_92f1_8d18a3c2ba62.slice/crio-b06f136084639eb54312e924b45a59034b2ff6df50bdf705ad19543a48440677 WatchSource:0}: Error finding container b06f136084639eb54312e924b45a59034b2ff6df50bdf705ad19543a48440677: Status 404 returned error can't find the container with id b06f136084639eb54312e924b45a59034b2ff6df50bdf705ad19543a48440677
Oct 30 21:25:12.441081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:12.441025    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:12.563508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:12.563465    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv" event={"ID":"7894ad8b-11d2-480f-92f1-8d18a3c2ba62","Type":"ContainerStarted","Data":"b06f136084639eb54312e924b45a59034b2ff6df50bdf705ad19543a48440677"}
Oct 30 21:25:12.735756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:12.735721    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" podUID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:25:12.735756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:12.735725    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" podUID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:25:13.317595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:13.317552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:13.571845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:13.571757    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv" event={"ID":"7894ad8b-11d2-480f-92f1-8d18a3c2ba62","Type":"ContainerStarted","Data":"7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85"}
Oct 30 21:25:13.571845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:13.571837    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:13.592646 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:13.592595    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv" podStartSLOduration=3.11933913 podStartE2EDuration="3.592579484s" totalImagesPullingTime="473.240354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:25:13.588990359 +0000 UTC m=+2821.703724549" watchObservedRunningTime="2026-10-30 21:25:13.592579484 +0000 UTC m=+2821.707313674"
Oct 30 21:25:14.053032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:14.052995    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:14.669330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:14.669271    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:14.897409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:14.885718    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:15.007410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:14.998043    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:15.248381 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:15.248290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:15.656641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:15.656599    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:15.959075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:15.958985    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:16.217466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:16.217244    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:17.019587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:17.019545    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:17.391880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:17.391786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:17.816638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:17.816602    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:18.740488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:18.740443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:19.480984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:19.480953    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:19.588635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:19.588595    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:25:20.107135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.107100    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:20.216688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.216653    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:20.241042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.241016    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:25:20.247894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.247868    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-plugins-config-volume\") pod \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") "
Oct 30 21:25:20.248026 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.247963    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-kube-api-access-6vzh9\" (UniqueName: \"kubernetes.io/projected/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-kube-api-access-6vzh9\") pod \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\" (UID: \"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff\") "
Oct 30 21:25:20.248275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.248256    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-plugins-config-volume" pod "cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" (UID: "cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:25:20.248611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.248593    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:20.250076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.250057    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-kube-api-access-6vzh9" pod "cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" (UID: "cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff"). InnerVolumeSpecName "kube-api-access-6vzh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:20.290516 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.280750    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:25:20.375410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.374602    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vzh9\" (UniqueName: \"kubernetes.io/projected/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff-kube-api-access-6vzh9\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:20.475838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.475753    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-plugins-config-volume\") pod \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") "
Oct 30 21:25:20.476020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.475951    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-kube-api-access-9wpzw\" (UniqueName: \"kubernetes.io/projected/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-kube-api-access-9wpzw\") pod \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\" (UID: \"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b\") "
Oct 30 21:25:20.476324 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.476300    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-plugins-config-volume" pod "38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" (UID: "38bb5b9c-3a30-416c-ab6d-8f9f8483c73b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:25:20.476744 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.476724    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:20.481643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.481609    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-kube-api-access-9wpzw" pod "38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" (UID: "38bb5b9c-3a30-416c-ab6d-8f9f8483c73b"). InnerVolumeSpecName "kube-api-access-9wpzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:20.578961 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.578925    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wpzw\" (UniqueName: \"kubernetes.io/projected/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b-kube-api-access-9wpzw\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:20.636805 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.636766    4852 generic.go:356] "Generic (PLEG): container finished" podID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" containerID="735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d" exitCode=0
Oct 30 21:25:20.637012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.636809    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" event={"ID":"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff","Type":"ContainerDied","Data":"735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d"}
Oct 30 21:25:20.637012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.636875    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q" event={"ID":"cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff","Type":"ContainerDied","Data":"9d9eeb5f68ec25a3d82f17d34b52f8e6cdbffa694f3f53082d908d6788092a68"}
Oct 30 21:25:20.637012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.636824    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"
Oct 30 21:25:20.637012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.636906    4852 scope.go:122] "RemoveContainer" containerID="735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d"
Oct 30 21:25:20.638538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.638509    4852 generic.go:356] "Generic (PLEG): container finished" podID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" containerID="fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324" exitCode=0
Oct 30 21:25:20.638820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.638584    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"
Oct 30 21:25:20.638820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.638607    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" event={"ID":"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b","Type":"ContainerDied","Data":"fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324"}
Oct 30 21:25:20.638820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.638687    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n" event={"ID":"38bb5b9c-3a30-416c-ab6d-8f9f8483c73b","Type":"ContainerDied","Data":"be9781bcb1e768244574b1c25bf1294aa874b6828d75e5c745f43ccae7e68ccd"}
Oct 30 21:25:20.662421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.662380    4852 scope.go:122] "RemoveContainer" containerID="735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d"
Oct 30 21:25:20.662770 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:25:20.662748    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d\": container with ID starting with 735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d not found: ID does not exist" containerID="735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d"
Oct 30 21:25:20.662873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.662778    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d"} err="failed to get container status \"735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d\": rpc error: code = NotFound desc = could not find container \"735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d\": container with ID starting with 735fa7423e2c366d8dbdb56da83a4bd1c454cd65f61db809ed696736df73154d not found: ID does not exist"
Oct 30 21:25:20.662873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.662794    4852 scope.go:122] "RemoveContainer" containerID="fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324"
Oct 30 21:25:20.673365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.673221    4852 scope.go:122] "RemoveContainer" containerID="fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324"
Oct 30 21:25:20.673974 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:25:20.673951    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324\": container with ID starting with fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324 not found: ID does not exist" containerID="fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324"
Oct 30 21:25:20.674076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.673993    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324"} err="failed to get container status \"fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324\": rpc error: code = NotFound desc = could not find container \"fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324\": container with ID starting with fc2ea2e8e63d7b0e48b9e2caa3f00425fd9abac14734f75cc018718da0515324 not found: ID does not exist"
Oct 30 21:25:20.678238 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.678206    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"]
Oct 30 21:25:20.684053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.684028    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-176db5-6ff4d88787-2xb9q"]
Oct 30 21:25:20.703002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.702970    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"]
Oct 30 21:25:20.707788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:20.707762    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-176db5-7d788cc755-kjj8n"]
Oct 30 21:25:21.180346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:21.180263    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:21.560033 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:25:21.559998    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:25:22.006560 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:22.006534    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" path="/var/lib/kubelet/pods/38bb5b9c-3a30-416c-ab6d-8f9f8483c73b/volumes"
Oct 30 21:25:22.006987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:22.006972    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" path="/var/lib/kubelet/pods/cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff/volumes"
Oct 30 21:25:22.119136 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:22.119099    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:22.947095 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:22.947056    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.041939 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:23.041679    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.163674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:23.163638    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.342958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:23.342924    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"]
Oct 30 21:25:23.343169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:23.343147    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy" containerID="cri-o://87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332" gracePeriod=30
Oct 30 21:25:23.433680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:23.433624    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:24.351938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:24.351901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:25.265575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:25.265537    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:26.036309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:26.036272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:26.228468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:26.228432    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:26.593276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:26.593242    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:26.695061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:26.695028    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:27.028244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:27.028213    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:27.818014 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:27.817982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:28.615833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.615797    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:28.650810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.650781    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:25:28.764879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.764835    4852 generic.go:356] "Generic (PLEG): container finished" podID="3683741c-abea-478c-bbfa-086679bda701" containerID="87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332" exitCode=0
Oct 30 21:25:28.765060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.764953    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"
Oct 30 21:25:28.765060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.764977    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" event={"ID":"3683741c-abea-478c-bbfa-086679bda701","Type":"ContainerDied","Data":"87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332"}
Oct 30 21:25:28.765060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.765035    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs" event={"ID":"3683741c-abea-478c-bbfa-086679bda701","Type":"ContainerDied","Data":"1cf878db97300696755ba1f05f7aaba02106e2de8747479934f82be85c1ed3cf"}
Oct 30 21:25:28.765246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.765066    4852 scope.go:122] "RemoveContainer" containerID="87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332"
Oct 30 21:25:28.779797 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.779776    4852 scope.go:122] "RemoveContainer" containerID="87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332"
Oct 30 21:25:28.780116 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:25:28.780094    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332\": container with ID starting with 87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332 not found: ID does not exist" containerID="87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332"
Oct 30 21:25:28.780213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.780126    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332"} err="failed to get container status \"87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332\": rpc error: code = NotFound desc = could not find container \"87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332\": container with ID starting with 87025a4d81f350bdc4db0c7c5b86cb6a0e4b17d5443c072c3b847bba9c641332 not found: ID does not exist"
Oct 30 21:25:28.804801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.804776    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-kube-api-access-8d7l9\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-kube-api-access-8d7l9\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.804933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.804810    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-socket\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.804974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.804950    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3683741c-abea-478c-bbfa-086679bda701-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3683741c-abea-478c-bbfa-086679bda701-istiod-ca-cert\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.805011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.804997    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-certs\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.805055 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805040    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-envoy\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.805143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805122    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-socket" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:28.805255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805236    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-istio-token\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-istio-token\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.805373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805285    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-data\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.805373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805284    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-certs" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:28.805373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805290    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3683741c-abea-478c-bbfa-086679bda701-istiod-ca-cert" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:25:28.805373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805331    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3683741c-abea-478c-bbfa-086679bda701-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3683741c-abea-478c-bbfa-086679bda701-istio-podinfo\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.805640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805557    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-credential-socket\") pod \"3683741c-abea-478c-bbfa-086679bda701\" (UID: \"3683741c-abea-478c-bbfa-086679bda701\") "
Oct 30 21:25:28.805640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805583    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-data" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:28.805887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.805862    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-credential-socket" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:28.806614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.806590    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.806699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.806620    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.806699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.806638    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3683741c-abea-478c-bbfa-086679bda701-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.806699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.806660    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.806699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.806674    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.807301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.807277    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-envoy" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:28.807424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.807340    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-kube-api-access-8d7l9" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "kube-api-access-8d7l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:28.807830 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.807803    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3683741c-abea-478c-bbfa-086679bda701-istio-podinfo" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:25:28.807924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.807839    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-istio-token" pod "3683741c-abea-478c-bbfa-086679bda701" (UID: "3683741c-abea-478c-bbfa-086679bda701"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:28.908716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.908630    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.908716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.908670    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3683741c-abea-478c-bbfa-086679bda701-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.908716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.908685    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d7l9\" (UniqueName: \"kubernetes.io/projected/3683741c-abea-478c-bbfa-086679bda701-kube-api-access-8d7l9\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.908716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:28.908699    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3683741c-abea-478c-bbfa-086679bda701-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:29.093936 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:29.093905    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"]
Oct 30 21:25:29.100842 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:29.100816    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-176db5-openshift-default-76d7ddc765-t7fbs"]
Oct 30 21:25:29.364357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:29.364319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:30.007036 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:30.006990    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3683741c-abea-478c-bbfa-086679bda701" path="/var/lib/kubelet/pods/3683741c-abea-478c-bbfa-086679bda701/volumes"
Oct 30 21:25:30.036895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:30.036856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:30.840331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:30.840287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:31.581860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:31.581828    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:32.551303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:32.551270    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:33.619734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:33.619702    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:34.084057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:34.084009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:34.193512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:34.193478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:34.770410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:34.764181    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.619288 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:35.619259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:36.702276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:36.702245    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:37.628892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:37.628861    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:38.388834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:38.388800    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:39.269345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:39.269299    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:40.147598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:40.147565    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:40.304627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:40.304587    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:40.739554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:40.739508    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:25:40.739772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:40.739576    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:25:40.972291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:40.972248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:41.714719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:41.714667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:41.810377 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:41.810333    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:42.721959 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:42.721926    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:43.536668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:43.536629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:44.018691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:44.018652    4852 generic.go:356] "Generic (PLEG): container finished" podID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerID="437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af" exitCode=0
Oct 30 21:25:44.019120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:44.018725    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" event={"ID":"5d9690d9-dfb2-47c7-805e-0a84c8a89b42","Type":"ContainerDied","Data":"437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af"}
Oct 30 21:25:44.019234 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:44.019217    4852 scope.go:122] "RemoveContainer" containerID="437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af"
Oct 30 21:25:44.021139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:44.021106    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:25:44.021235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:44.021177    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:25:44.021235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:44.021214    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:25:44.352526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:44.352444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:45.028444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:45.028411    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" event={"ID":"5d9690d9-dfb2-47c7-805e-0a84c8a89b42","Type":"ContainerStarted","Data":"e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19"}
Oct 30 21:25:45.073514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:45.072687    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 21:25:45.073514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:45.072925    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 21:25:45.073909 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:45.073829    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:25:45.228707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:45.228667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.035168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:46.035133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.075203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:46.074896    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 21:25:46.075203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:46.074961    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 21:25:46.298294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:46.298208    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.467772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:46.467735    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.816761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:46.816724    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.981294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:46.981264    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:47.064412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:47.063263    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 21:25:47.064412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:47.063334    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 21:25:47.817542 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:47.817500    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:48.061423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:48.060862    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 21:25:48.061423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:48.060940    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 21:25:48.859420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:48.859289    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:49.080415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:49.078792    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 21:25:49.080415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:49.078855    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 21:25:49.875551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:49.868975    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:50.073414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:50.072899    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 30 21:25:50.073414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:50.072957    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 30 21:25:50.913735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:50.913700    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:51.065195 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:51.065162    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:25:51.065663 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:51.065650    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:25:51.066398 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:51.066370    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:25:51.560328 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:25:51.560293    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:25:51.654623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:51.654574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:52.423249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:52.423191    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:52.505355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:52.505312    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:53.424634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:53.424595    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:53.901760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:53.901728    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.005394 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:54.005343    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.215287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:54.215204    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"]
Oct 30 21:25:54.215736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:54.215699    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy" containerID="cri-o://35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c" gracePeriod=30
Oct 30 21:25:54.249677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:54.249637    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.367411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:54.360942    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.723795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:54.723757    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:55.464551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:55.464455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:56.159112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:56.159072    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:57.019092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:57.019045    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:57.825446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:57.825405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:58.411087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:58.411054    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:25:58.411310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:58.411114    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:25:58.618834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:58.618795    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.279979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.279945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.368910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.368869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.729496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.729235    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"]
Oct 30 21:25:59.745739 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.745708    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:25:59.746966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.746933    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.759465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.759437    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"]
Oct 30 21:25:59.765726 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.765704    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"]
Oct 30 21:25:59.789113 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.788714    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-socket\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.789113 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.788797    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-envoy\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.789113 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.788910    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-podinfo\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.789113 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.788963    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istiod-ca-cert\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.791462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.789125    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-certs\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.791462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.789155    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-kube-api-access-qk6st\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-kube-api-access-qk6st\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.791462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.789208    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-credential-socket\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.791462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.789277    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-token\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-token\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.791462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.789334    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-data\") pod \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\" (UID: \"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1\") "
Oct 30 21:25:59.791462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.790902    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-socket" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:59.791462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.791364    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istiod-ca-cert" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:25:59.792587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.791862    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-data" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:59.792587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.792147    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-certs" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:59.793310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.793275    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-credential-socket" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:59.796452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.796370    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-podinfo" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:25:59.807991 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.807933    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-envoy" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:25:59.807991 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.807961    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-kube-api-access-qk6st" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "kube-api-access-qk6st". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:59.809673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.809643    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-token" pod "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" (UID: "2f3bf68a-44ac-4f67-a4f6-8e79c517bac1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:59.845368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.845336    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.891598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891561    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891592    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891602    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891614    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891623    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk6st\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-kube-api-access-qk6st\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891633    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891643    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891653    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:59.891849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:25:59.891662    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:00.101257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.101218    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:00.282893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.276613    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:00.360719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.359857    4852 generic.go:356] "Generic (PLEG): container finished" podID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerID="35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c" exitCode=0
Oct 30 21:26:00.360719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.359971    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" event={"ID":"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1","Type":"ContainerDied","Data":"35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c"}
Oct 30 21:26:00.360719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.360024    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v" event={"ID":"2f3bf68a-44ac-4f67-a4f6-8e79c517bac1","Type":"ContainerDied","Data":"85cec5e0e43a39fb6efca6ecaca1d94d8f63ab5bd857e987715557aeeb697a82"}
Oct 30 21:26:00.360719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.360058    4852 scope.go:122] "RemoveContainer" containerID="35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c"
Oct 30 21:26:00.360719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.360275    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"
Oct 30 21:26:00.416272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.416231    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"]
Oct 30 21:26:00.418490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.418464    4852 scope.go:122] "RemoveContainer" containerID="58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182"
Oct 30 21:26:00.424501 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.424296    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9f4-openshift-default-5dcf868997-8sg6v"]
Oct 30 21:26:00.431246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.431219    4852 scope.go:122] "RemoveContainer" containerID="35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c"
Oct 30 21:26:00.431590 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:00.431570    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c\": container with ID starting with 35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c not found: ID does not exist" containerID="35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c"
Oct 30 21:26:00.431651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.431598    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c"} err="failed to get container status \"35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c\": rpc error: code = NotFound desc = could not find container \"35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c\": container with ID starting with 35c04d6732b229df0c0761f64c4293827e1efd291a24f7ce874e976458d1dc3c not found: ID does not exist"
Oct 30 21:26:00.431651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.431616    4852 scope.go:122] "RemoveContainer" containerID="58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182"
Oct 30 21:26:00.432025 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:00.431935    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182\": container with ID starting with 58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182 not found: ID does not exist" containerID="58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182"
Oct 30 21:26:00.432107 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.432032    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182"} err="failed to get container status \"58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182\": rpc error: code = NotFound desc = could not find container \"58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182\": container with ID starting with 58239bf7b3c9efe9c8f45aadd5fddfc9fe48ce8e072ee5a6edbb7168acbb6182 not found: ID does not exist"
Oct 30 21:26:00.826645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:00.826601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:01.434178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:01.434134    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:01.531046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:01.531012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:01.640203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:01.640144    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:02.006159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:02.006127    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" path="/var/lib/kubelet/pods/2f3bf68a-44ac-4f67-a4f6-8e79c517bac1/volumes"
Oct 30 21:26:02.354475 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:02.353791    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:03.151833 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:03.151748    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:03.921086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:03.921055    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:04.604311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:04.604275    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:05.038322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:05.038285    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"]
Oct 30 21:26:05.054326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:05.054289    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"]
Oct 30 21:26:05.103945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:05.103913    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:05.203656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:05.203622    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:05.404865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:05.404782    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:05.505272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:05.505235    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:06.219907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:06.219876    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:06.940990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:06.940955    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:07.140360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:07.140324    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:07.223042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:07.222921    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:07.577371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:07.577327    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:07.699780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:07.699751    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:08.012415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:08.012362    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:08.343839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:08.343697    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:08.714668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:08.714571    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:09.532223 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:09.532186    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.039247 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:10.039206    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" podUID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" containerName="payload-processing" containerID="cri-o://2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338" gracePeriod=25
Oct 30 21:26:10.055039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:10.054999    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" podUID="73b5e064-066d-40f1-badb-15b3f5d9a641" containerName="payload-pre-processing" containerID="cri-o://1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39" gracePeriod=25
Oct 30 21:26:10.422479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:10.419204    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:11.331460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:11.331376    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:11.424503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:11.424473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:12.457360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.457317    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:12.732466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.732277    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:12.845234 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.845195    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:12.846237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.846208    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"]
Oct 30 21:26:12.847044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847021    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy"
Oct 30 21:26:12.847044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847040    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847059    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" containerName="payload-processing"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847067    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" containerName="payload-processing"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847081    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847087    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847107    4852 cpu_manager.go:414] "containerMap: removing container" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847111    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847121    4852 cpu_manager.go:414] "containerMap: removing container" podUID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" containerName="payload-pre-processing"
Oct 30 21:26:12.847212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847125    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" containerName="payload-pre-processing"
Oct 30 21:26:12.847714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847257    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy"
Oct 30 21:26:12.847714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847267    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf7e6386-96d1-42cc-b5d0-a59ed2f2dfff" containerName="payload-processing"
Oct 30 21:26:12.847714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847277    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f3bf68a-44ac-4f67-a4f6-8e79c517bac1" containerName="istio-proxy"
Oct 30 21:26:12.847714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847284    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="3683741c-abea-478c-bbfa-086679bda701" containerName="istio-proxy"
Oct 30 21:26:12.847714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.847292    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="38bb5b9c-3a30-416c-ab6d-8f9f8483c73b" containerName="payload-pre-processing"
Oct 30 21:26:12.864937 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.864903    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:12.872180 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.872145    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"]
Oct 30 21:26:12.872368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.872270    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-0c4052-dockercfg-n4bdr\""
Oct 30 21:26:12.872612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.872589    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-0c4052\""
Oct 30 21:26:12.941876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.941839    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0335594b-ba42-4f48-96d0-3196c63389ad-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:12.942092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:12.941956    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp2xm\" (UniqueName: \"kubernetes.io/projected/0335594b-ba42-4f48-96d0-3196c63389ad-kube-api-access-cp2xm\") pod \"payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:13.058502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:13.058462    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2xm\" (UniqueName: \"kubernetes.io/projected/0335594b-ba42-4f48-96d0-3196c63389ad-kube-api-access-cp2xm\") pod \"payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:13.094446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:13.080181    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp2xm\" (UniqueName: \"kubernetes.io/projected/0335594b-ba42-4f48-96d0-3196c63389ad-kube-api-access-cp2xm\") pod \"payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:13.095172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:13.095143    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0335594b-ba42-4f48-96d0-3196c63389ad-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:13.096284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:13.096256    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0335594b-ba42-4f48-96d0-3196c63389ad-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:13.206715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:13.206683    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:13.630982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:13.630945    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"]
Oct 30 21:26:13.631925 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:26:13.631898    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0335594b_ba42_4f48_96d0_3196c63389ad.slice/crio-0de8778623871d94bf9c0603db199d9bb55ba53caf16ace2fcc743250e395eea WatchSource:0}: Error finding container 0de8778623871d94bf9c0603db199d9bb55ba53caf16ace2fcc743250e395eea: Status 404 returned error can't find the container with id 0de8778623871d94bf9c0603db199d9bb55ba53caf16ace2fcc743250e395eea
Oct 30 21:26:13.858813 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:13.858778    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:14.579536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.579498    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974" event={"ID":"0335594b-ba42-4f48-96d0-3196c63389ad","Type":"ContainerStarted","Data":"2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9"}
Oct 30 21:26:14.579731 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.579555    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974" event={"ID":"0335594b-ba42-4f48-96d0-3196c63389ad","Type":"ContainerStarted","Data":"0de8778623871d94bf9c0603db199d9bb55ba53caf16ace2fcc743250e395eea"}
Oct 30 21:26:14.579731 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.579603    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:14.600266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.600215    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974" podStartSLOduration=2.600202679 podStartE2EDuration="2.600202679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:14.595692695 +0000 UTC m=+2882.710426885" watchObservedRunningTime="2026-10-30 21:26:14.600202679 +0000 UTC m=+2882.714936869"
Oct 30 21:26:14.730796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.730715    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:14.768302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.768255    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" containerID="cri-o://2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97" gracePeriod=45
Oct 30 21:26:14.828409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.820620    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main" containerID="cri-o://cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860" gracePeriod=45
Oct 30 21:26:14.828409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:14.820773    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" containerID="cri-o://a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c" gracePeriod=45
Oct 30 21:26:15.145334 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.144991    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:26:15.154766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.153792    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1fa0a83b-ab79-4a98-8030-1b05158342d1-kube-api-access-zdkdz\" (UniqueName: \"kubernetes.io/projected/1fa0a83b-ab79-4a98-8030-1b05158342d1-kube-api-access-zdkdz\") pod \"1fa0a83b-ab79-4a98-8030-1b05158342d1\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") "
Oct 30 21:26:15.154766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.153925    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-model-cache\") pod \"1fa0a83b-ab79-4a98-8030-1b05158342d1\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") "
Oct 30 21:26:15.154766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.154001    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-dshm\") pod \"1fa0a83b-ab79-4a98-8030-1b05158342d1\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") "
Oct 30 21:26:15.154766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.154118    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-tmp-dir\") pod \"1fa0a83b-ab79-4a98-8030-1b05158342d1\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") "
Oct 30 21:26:15.154766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.154168    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1fa0a83b-ab79-4a98-8030-1b05158342d1-tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa0a83b-ab79-4a98-8030-1b05158342d1-tls-certs\") pod \"1fa0a83b-ab79-4a98-8030-1b05158342d1\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") "
Oct 30 21:26:15.154766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.154200    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-home\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-home\") pod \"1fa0a83b-ab79-4a98-8030-1b05158342d1\" (UID: \"1fa0a83b-ab79-4a98-8030-1b05158342d1\") "
Oct 30 21:26:15.155176 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.154988    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-home" pod "1fa0a83b-ab79-4a98-8030-1b05158342d1" (UID: "1fa0a83b-ab79-4a98-8030-1b05158342d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.155176 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.155142    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-tmp-dir" pod "1fa0a83b-ab79-4a98-8030-1b05158342d1" (UID: "1fa0a83b-ab79-4a98-8030-1b05158342d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.156599 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.156255    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-model-cache" pod "1fa0a83b-ab79-4a98-8030-1b05158342d1" (UID: "1fa0a83b-ab79-4a98-8030-1b05158342d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.159036 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.159016    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:26:15.163843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.163819    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-dshm" pod "1fa0a83b-ab79-4a98-8030-1b05158342d1" (UID: "1fa0a83b-ab79-4a98-8030-1b05158342d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.165283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.165257    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa0a83b-ab79-4a98-8030-1b05158342d1-tls-certs" pod "1fa0a83b-ab79-4a98-8030-1b05158342d1" (UID: "1fa0a83b-ab79-4a98-8030-1b05158342d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:26:15.173368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.171008    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa0a83b-ab79-4a98-8030-1b05158342d1-kube-api-access-zdkdz" pod "1fa0a83b-ab79-4a98-8030-1b05158342d1" (UID: "1fa0a83b-ab79-4a98-8030-1b05158342d1"). InnerVolumeSpecName "kube-api-access-zdkdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:15.173638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.173614    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:26:15.255827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.255787    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-home\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-home\") pod \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") "
Oct 30 21:26:15.256001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.255909    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tmp-dir\") pod \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") "
Oct 30 21:26:15.256001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.255951    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tls-certs\") pod \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") "
Oct 30 21:26:15.256001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.255987    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0540acbd-331d-4277-b186-acf474a45624-kube-api-access-t2pt4\" (UniqueName: \"kubernetes.io/projected/0540acbd-331d-4277-b186-acf474a45624-kube-api-access-t2pt4\") pod \"0540acbd-331d-4277-b186-acf474a45624\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") "
Oct 30 21:26:15.256123 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256014    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-dshm\") pod \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") "
Oct 30 21:26:15.256123 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256100    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0540acbd-331d-4277-b186-acf474a45624-tls-certs\" (UniqueName: \"kubernetes.io/secret/0540acbd-331d-4277-b186-acf474a45624-tls-certs\") pod \"0540acbd-331d-4277-b186-acf474a45624\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") "
Oct 30 21:26:15.256242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256162    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-home\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-home\") pod \"0540acbd-331d-4277-b186-acf474a45624\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") "
Oct 30 21:26:15.256242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256210    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-model-cache\") pod \"0540acbd-331d-4277-b186-acf474a45624\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") "
Oct 30 21:26:15.256365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256277    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-model-cache\") pod \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") "
Oct 30 21:26:15.256365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256275    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-home" pod "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" (UID: "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.256365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256306    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-kube-api-access-bqlj2\" (UniqueName: \"kubernetes.io/projected/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-kube-api-access-bqlj2\") pod \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\" (UID: \"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e\") "
Oct 30 21:26:15.256602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256444    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-dshm\") pod \"0540acbd-331d-4277-b186-acf474a45624\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") "
Oct 30 21:26:15.256602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256479    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-tmp-dir\") pod \"0540acbd-331d-4277-b186-acf474a45624\" (UID: \"0540acbd-331d-4277-b186-acf474a45624\") "
Oct 30 21:26:15.256602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256486    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tmp-dir" pod "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" (UID: "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.257931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256673    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-home" pod "0540acbd-331d-4277-b186-acf474a45624" (UID: "0540acbd-331d-4277-b186-acf474a45624"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.257931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.256781    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-model-cache" pod "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" (UID: "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.257931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257170    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-model-cache" pod "0540acbd-331d-4277-b186-acf474a45624" (UID: "0540acbd-331d-4277-b186-acf474a45624"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.257931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257812    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-tmp-dir" pod "0540acbd-331d-4277-b186-acf474a45624" (UID: "0540acbd-331d-4277-b186-acf474a45624"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.257931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257586    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.258365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257958    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.258365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257969    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fa0a83b-ab79-4a98-8030-1b05158342d1-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.258365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257977    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.258365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257989    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdkdz\" (UniqueName: \"kubernetes.io/projected/1fa0a83b-ab79-4a98-8030-1b05158342d1-kube-api-access-zdkdz\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.258365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.257998    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.258365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.258006    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fa0a83b-ab79-4a98-8030-1b05158342d1-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.259698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.259638    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-dshm" pod "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" (UID: "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.259940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.259918    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0540acbd-331d-4277-b186-acf474a45624-tls-certs" pod "0540acbd-331d-4277-b186-acf474a45624" (UID: "0540acbd-331d-4277-b186-acf474a45624"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:26:15.260014 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.259988    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tls-certs" pod "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" (UID: "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:26:15.260111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.260094    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0540acbd-331d-4277-b186-acf474a45624-kube-api-access-t2pt4" pod "0540acbd-331d-4277-b186-acf474a45624" (UID: "0540acbd-331d-4277-b186-acf474a45624"). InnerVolumeSpecName "kube-api-access-t2pt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:15.260704 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.260682    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-dshm" pod "0540acbd-331d-4277-b186-acf474a45624" (UID: "0540acbd-331d-4277-b186-acf474a45624"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:15.260960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.260933    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-kube-api-access-bqlj2" pod "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" (UID: "ea0d2309-4b0c-4d95-8ff8-886b163bbe7e"). InnerVolumeSpecName "kube-api-access-bqlj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:15.360139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360093    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360129    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360143    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360157    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqlj2\" (UniqueName: \"kubernetes.io/projected/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-kube-api-access-bqlj2\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360172    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360183    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0540acbd-331d-4277-b186-acf474a45624-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360195    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360206    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360217    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2pt4\" (UniqueName: \"kubernetes.io/projected/0540acbd-331d-4277-b186-acf474a45624-kube-api-access-t2pt4\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360230    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.360445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.360242    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0540acbd-331d-4277-b186-acf474a45624-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:15.589709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.589609    4852 generic.go:356] "Generic (PLEG): container finished" podID="0540acbd-331d-4277-b186-acf474a45624" containerID="a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c" exitCode=0
Oct 30 21:26:15.589709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.589625    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:15.589966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.589710    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"
Oct 30 21:26:15.589966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.589738    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" event={"ID":"0540acbd-331d-4277-b186-acf474a45624","Type":"ContainerDied","Data":"a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c"}
Oct 30 21:26:15.589966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.589774    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" event={"ID":"0540acbd-331d-4277-b186-acf474a45624","Type":"ContainerDied","Data":"3019b8aced0b55a25cb28385c8386fbd94901e3e09031c0b94d342520c5bdfb9"}
Oct 30 21:26:15.589966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.589795    4852 scope.go:122] "RemoveContainer" containerID="a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c"
Oct 30 21:26:15.591329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.591305    4852 generic.go:356] "Generic (PLEG): container finished" podID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerID="2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97" exitCode=0
Oct 30 21:26:15.591435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.591357    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"
Oct 30 21:26:15.591506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.591411    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" event={"ID":"1fa0a83b-ab79-4a98-8030-1b05158342d1","Type":"ContainerDied","Data":"2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97"}
Oct 30 21:26:15.591506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.591492    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" event={"ID":"1fa0a83b-ab79-4a98-8030-1b05158342d1","Type":"ContainerDied","Data":"f25315ff07f3cdf829ba4b0508711c73d273ad22b34c23959397a0f4243e4898"}
Oct 30 21:26:15.594429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.593260    4852 generic.go:356] "Generic (PLEG): container finished" podID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerID="cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860" exitCode=0
Oct 30 21:26:15.594429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.593326    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"
Oct 30 21:26:15.594429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.593344    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" event={"ID":"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e","Type":"ContainerDied","Data":"cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860"}
Oct 30 21:26:15.594429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.593400    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph" event={"ID":"ea0d2309-4b0c-4d95-8ff8-886b163bbe7e","Type":"ContainerDied","Data":"87b0299e76254fe3878b01037d1fd764886f29d1f688114cc22697b164ae0f3f"}
Oct 30 21:26:15.602453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.602259    4852 scope.go:122] "RemoveContainer" containerID="a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c"
Oct 30 21:26:15.602592 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:15.602570    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c\": container with ID starting with a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c not found: ID does not exist" containerID="a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c"
Oct 30 21:26:15.602671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.602601    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c"} err="failed to get container status \"a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c\": rpc error: code = NotFound desc = could not find container \"a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c\": container with ID starting with a4c41c8be5ac1a068059bcf2dcb5dce85a56ca10a641a62d4b30dcbb99b0762c not found: ID does not exist"
Oct 30 21:26:15.602671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.602616    4852 scope.go:122] "RemoveContainer" containerID="2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97"
Oct 30 21:26:15.613779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.613743    4852 scope.go:122] "RemoveContainer" containerID="2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97"
Oct 30 21:26:15.614221 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:15.614190    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97\": container with ID starting with 2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97 not found: ID does not exist" containerID="2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97"
Oct 30 21:26:15.614364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.614235    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97"} err="failed to get container status \"2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97\": rpc error: code = NotFound desc = could not find container \"2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97\": container with ID starting with 2ef13bbad84dc902b4396624ee6f39571df3815dee6e551a12980533be073f97 not found: ID does not exist"
Oct 30 21:26:15.614364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.614261    4852 scope.go:122] "RemoveContainer" containerID="cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860"
Oct 30 21:26:15.624626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.624603    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"]
Oct 30 21:26:15.629297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.629272    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z"]
Oct 30 21:26:15.633738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.633702    4852 scope.go:122] "RemoveContainer" containerID="cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860"
Oct 30 21:26:15.634244 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:15.634067    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860\": container with ID starting with cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860 not found: ID does not exist" containerID="cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860"
Oct 30 21:26:15.634244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.634102    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860"} err="failed to get container status \"cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860\": rpc error: code = NotFound desc = could not find container \"cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860\": container with ID starting with cff69ec0f0e97fdc82b30d0baf6365622edb4aa4eb022c85e36c512513b1e860 not found: ID does not exist"
Oct 30 21:26:15.639431 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.639410    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"]
Oct 30 21:26:15.644314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.643553    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57"]
Oct 30 21:26:15.701467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.701365    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"]
Oct 30 21:26:15.707306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:15.707274    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-8e4acb/distinct-w2-8e4acb-kserve-5ddc7cfbdb-jhgph"]
Oct 30 21:26:16.007587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:16.007559    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0540acbd-331d-4277-b186-acf474a45624" path="/var/lib/kubelet/pods/0540acbd-331d-4277-b186-acf474a45624/volumes"
Oct 30 21:26:16.008088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:16.008070    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" path="/var/lib/kubelet/pods/1fa0a83b-ab79-4a98-8030-1b05158342d1/volumes"
Oct 30 21:26:16.008539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:16.008525    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" path="/var/lib/kubelet/pods/ea0d2309-4b0c-4d95-8ff8-886b163bbe7e/volumes"
Oct 30 21:26:16.097759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:16.097715    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/simulator-w2-8e4acb-kserve-74d47f8cc4-psn7z" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/ready\": context deadline exceeded"
Oct 30 21:26:16.109963 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:16.109922    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-8e4acb/distinct-2-w2-8e4acb-kserve-7f98cb4c85-kxm57" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:16.430839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:16.430749    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:17.333379 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:17.333345    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:18.132773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:18.132733    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:18.951695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:18.951654    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:19.725655 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:19.725615    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:19.794038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:19.793994    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" podUID="73b5e064-066d-40f1-badb-15b3f5d9a641" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:26:19.794687 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:19.794650    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" podUID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:26:20.527686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:20.527648    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:21.209906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:21.209872    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:21.560813 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:21.560774    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:26:21.939752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:21.939680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:22.662665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:22.662623    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:23.353257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:23.353222    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:24.136016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:24.135981    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:24.879545 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:24.879506    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:25.597323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:25.597295    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:26:25.607866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:25.607828    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:26.432504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:26.432462    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:27.139510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:27.139473    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:27.900267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:27.900223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:28.667878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:28.667842    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:29.529499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:29.529447    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:29.787898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:29.787805    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" podUID="73b5e064-066d-40f1-badb-15b3f5d9a641" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:26:29.787898 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:29.787821    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" podUID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:26:30.220550 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.220484    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:30.448775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.448745    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 21:26:30.472134 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.472099    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 21:26:30.534856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.534814    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7b0562c-6755-48c3-a8df-c26a16ad4c22-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b0562c-6755-48c3-a8df-c26a16ad4c22-plugins-config-volume\") pod \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") "
Oct 30 21:26:30.535058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.534875    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73b5e064-066d-40f1-badb-15b3f5d9a641-kube-api-access-rnms7\" (UniqueName: \"kubernetes.io/projected/73b5e064-066d-40f1-badb-15b3f5d9a641-kube-api-access-rnms7\") pod \"73b5e064-066d-40f1-badb-15b3f5d9a641\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") "
Oct 30 21:26:30.535058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.534914    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73b5e064-066d-40f1-badb-15b3f5d9a641-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b5e064-066d-40f1-badb-15b3f5d9a641-plugins-config-volume\") pod \"73b5e064-066d-40f1-badb-15b3f5d9a641\" (UID: \"73b5e064-066d-40f1-badb-15b3f5d9a641\") "
Oct 30 21:26:30.535058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.534948    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7b0562c-6755-48c3-a8df-c26a16ad4c22-kube-api-access-6hlws\" (UniqueName: \"kubernetes.io/projected/e7b0562c-6755-48c3-a8df-c26a16ad4c22-kube-api-access-6hlws\") pod \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\" (UID: \"e7b0562c-6755-48c3-a8df-c26a16ad4c22\") "
Oct 30 21:26:30.535337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.535237    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b0562c-6755-48c3-a8df-c26a16ad4c22-plugins-config-volume" pod "e7b0562c-6755-48c3-a8df-c26a16ad4c22" (UID: "e7b0562c-6755-48c3-a8df-c26a16ad4c22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:30.535686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.535660    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73b5e064-066d-40f1-badb-15b3f5d9a641-plugins-config-volume" pod "73b5e064-066d-40f1-badb-15b3f5d9a641" (UID: "73b5e064-066d-40f1-badb-15b3f5d9a641"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:30.535826 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.535808    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b0562c-6755-48c3-a8df-c26a16ad4c22-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:30.535903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.535831    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b5e064-066d-40f1-badb-15b3f5d9a641-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:30.537570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.537547    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7b0562c-6755-48c3-a8df-c26a16ad4c22-kube-api-access-6hlws" pod "e7b0562c-6755-48c3-a8df-c26a16ad4c22" (UID: "e7b0562c-6755-48c3-a8df-c26a16ad4c22"). InnerVolumeSpecName "kube-api-access-6hlws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:30.537920 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.537904    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b5e064-066d-40f1-badb-15b3f5d9a641-kube-api-access-rnms7" pod "73b5e064-066d-40f1-badb-15b3f5d9a641" (UID: "73b5e064-066d-40f1-badb-15b3f5d9a641"). InnerVolumeSpecName "kube-api-access-rnms7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:30.637356 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.637269    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnms7\" (UniqueName: \"kubernetes.io/projected/73b5e064-066d-40f1-badb-15b3f5d9a641-kube-api-access-rnms7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:30.637356 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.637304    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hlws\" (UniqueName: \"kubernetes.io/projected/e7b0562c-6755-48c3-a8df-c26a16ad4c22-kube-api-access-6hlws\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:30.792121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.792089    4852 generic.go:356] "Generic (PLEG): container finished" podID="73b5e064-066d-40f1-badb-15b3f5d9a641" containerID="1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39" exitCode=0
Oct 30 21:26:30.792721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.792138    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"
Oct 30 21:26:30.792721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.792178    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" event={"ID":"73b5e064-066d-40f1-badb-15b3f5d9a641","Type":"ContainerDied","Data":"1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39"}
Oct 30 21:26:30.792721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.792229    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc" event={"ID":"73b5e064-066d-40f1-badb-15b3f5d9a641","Type":"ContainerDied","Data":"3b43c8410dc54dc974a1906cf38db6ac1ed85b78e15616436e58867b29100d0f"}
Oct 30 21:26:30.792721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.792252    4852 scope.go:122] "RemoveContainer" containerID="1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39"
Oct 30 21:26:30.793812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.793785    4852 generic.go:356] "Generic (PLEG): container finished" podID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" containerID="2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338" exitCode=0
Oct 30 21:26:30.793940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.793840    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"
Oct 30 21:26:30.793940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.793840    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" event={"ID":"e7b0562c-6755-48c3-a8df-c26a16ad4c22","Type":"ContainerDied","Data":"2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338"}
Oct 30 21:26:30.793940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.793897    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m" event={"ID":"e7b0562c-6755-48c3-a8df-c26a16ad4c22","Type":"ContainerDied","Data":"03f03b236e17b30e5a4e93e2ffa346c64b34b10a7cb3279dd96893e4fe8ce6a6"}
Oct 30 21:26:30.805579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.805557    4852 scope.go:122] "RemoveContainer" containerID="1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39"
Oct 30 21:26:30.805893 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:30.805873    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39\": container with ID starting with 1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39 not found: ID does not exist" containerID="1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39"
Oct 30 21:26:30.805993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.805905    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39"} err="failed to get container status \"1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39\": rpc error: code = NotFound desc = could not find container \"1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39\": container with ID starting with 1a99e56797f6445dae06a38f4ee35656a1989661d071e3da28c2eb647f571b39 not found: ID does not exist"
Oct 30 21:26:30.805993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.805927    4852 scope.go:122] "RemoveContainer" containerID="2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338"
Oct 30 21:26:30.819513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.819473    4852 scope.go:122] "RemoveContainer" containerID="2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338"
Oct 30 21:26:30.819878 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:30.819854    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338\": container with ID starting with 2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338 not found: ID does not exist" containerID="2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338"
Oct 30 21:26:30.819978 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.819892    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338"} err="failed to get container status \"2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338\": rpc error: code = NotFound desc = could not find container \"2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338\": container with ID starting with 2edca34d3dc50f17ce0210e3b9cd605b397af0b2b6f7f543f6904b222079b338 not found: ID does not exist"
Oct 30 21:26:30.820840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.820817    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"]
Oct 30 21:26:30.825314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.825284    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-8e4acb-7b65d5967f-t74mc"]
Oct 30 21:26:30.842800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.842737    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"]
Oct 30 21:26:30.849433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.849380    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-8e4acb-9797567d5-l5k7m"]
Oct 30 21:26:30.873239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.873203    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"]
Oct 30 21:26:30.874085 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874062    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main"
Oct 30 21:26:30.874085 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874084    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874111    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" containerName="payload-processing"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874118    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" containerName="payload-processing"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874136    4852 cpu_manager.go:414] "containerMap: removing container" podUID="73b5e064-066d-40f1-badb-15b3f5d9a641" containerName="payload-pre-processing"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874143    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73b5e064-066d-40f1-badb-15b3f5d9a641" containerName="payload-pre-processing"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874173    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874180    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874209    4852 cpu_manager.go:414] "containerMap: removing container" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main"
Oct 30 21:26:30.874294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874217    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main"
Oct 30 21:26:30.875244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874407    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="1fa0a83b-ab79-4a98-8030-1b05158342d1" containerName="main"
Oct 30 21:26:30.875244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874429    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0540acbd-331d-4277-b186-acf474a45624" containerName="main"
Oct 30 21:26:30.875244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874442    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" containerName="payload-processing"
Oct 30 21:26:30.875244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874451    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="73b5e064-066d-40f1-badb-15b3f5d9a641" containerName="payload-pre-processing"
Oct 30 21:26:30.875244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.874463    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea0d2309-4b0c-4d95-8ff8-886b163bbe7e" containerName="main"
Oct 30 21:26:30.880082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.880044    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:30.905454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.905363    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0c4052-serving-cert\""
Oct 30 21:26:30.906838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.906811    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"]
Oct 30 21:26:30.923468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.923439    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"]
Oct 30 21:26:30.928168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.928144    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.943901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.941687    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:30.943901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.943352    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"]
Oct 30 21:26:30.944089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.943955    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-token\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.943977    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istiod-ca-cert\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944064    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-data\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944090    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4t7x\" (UniqueName: \"kubernetes.io/projected/b69450b8-9338-46e2-9c5a-094fad128ea4-kube-api-access-g4t7x\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:30.944236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944118    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-metrics-certs\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:30.944350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944319    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69450b8-9338-46e2-9c5a-094fad128ea4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:30.944469 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944420    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:30.944469 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944456    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-envoy\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944483    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-certs\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944531    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-podinfo\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944610    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-credential-socket\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944870    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-socket\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.944959 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.944912    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spzvp\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-kube-api-access-spzvp\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:30.989209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:30.989176    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a3fb70-openshift-default-dockercfg-clgk4\""
Oct 30 21:26:31.051996 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.051884    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-data\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.052273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.052247    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4t7x\" (UniqueName: \"kubernetes.io/projected/b69450b8-9338-46e2-9c5a-094fad128ea4-kube-api-access-g4t7x\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.052434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.052416    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-metrics-certs\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.052652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.052637    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69450b8-9338-46e2-9c5a-094fad128ea4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.052821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.052803    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.052955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.052937    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-envoy\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.053065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.053051    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-certs\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.053176 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.053163    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-podinfo\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.053310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.053297    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-credential-socket\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.053521 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.053504    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-socket\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.053654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.053640    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spzvp\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-kube-api-access-spzvp\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.053799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.053781    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-token\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.053918 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.053902    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istiod-ca-cert\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.055154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.055129    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istiod-ca-cert\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.058012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.057274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:31.064806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.063691    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-envoy\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.064806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.064062    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-data\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.070812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.065698    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-certs\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.070812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.066326    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-socket\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.070812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.066592    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-credential-socket\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.070812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.067597    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69450b8-9338-46e2-9c5a-094fad128ea4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.119423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.111301    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-token\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.119423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.112382    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4t7x\" (UniqueName: \"kubernetes.io/projected/b69450b8-9338-46e2-9c5a-094fad128ea4-kube-api-access-g4t7x\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.119423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.113311    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-metrics-certs\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.136568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.123533    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spzvp\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-kube-api-access-spzvp\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.136568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.130288    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-podinfo\") pod \"e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") " pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.136568 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.131032    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-maas-api-tls\") pod \"maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.219699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.219597    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:31.248928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.248897    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:31.619638 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.619602    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"]
Oct 30 21:26:31.637650 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.637603    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"]
Oct 30 21:26:31.648229 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:26:31.648198    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ec5fb08_ff18_48b2_9611_eee1690dbb07.slice/crio-f74d739f14ee2400efc2278a6a3df8c8d1fbc492cbfa7c017d771509953bb5ab WatchSource:0}: Error finding container f74d739f14ee2400efc2278a6a3df8c8d1fbc492cbfa7c017d771509953bb5ab: Status 404 returned error can't find the container with id f74d739f14ee2400efc2278a6a3df8c8d1fbc492cbfa7c017d771509953bb5ab
Oct 30 21:26:31.656483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.656433    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:26:31.656483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.656539    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:26:31.656483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.656586    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:26:31.806064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.806025    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5" event={"ID":"b69450b8-9338-46e2-9c5a-094fad128ea4","Type":"ContainerStarted","Data":"ebd5c8a57e775525c5676f6fbb259f56efc52e63732d3d10ff4d0520807634be"}
Oct 30 21:26:31.808090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.808055    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" event={"ID":"0ec5fb08-ff18-48b2-9611-eee1690dbb07","Type":"ContainerStarted","Data":"649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68"}
Oct 30 21:26:31.808257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.808111    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" event={"ID":"0ec5fb08-ff18-48b2-9611-eee1690dbb07","Type":"ContainerStarted","Data":"f74d739f14ee2400efc2278a6a3df8c8d1fbc492cbfa7c017d771509953bb5ab"}
Oct 30 21:26:31.940782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:31.940688    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:32.091355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.091319    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73b5e064-066d-40f1-badb-15b3f5d9a641" path="/var/lib/kubelet/pods/73b5e064-066d-40f1-badb-15b3f5d9a641/volumes"
Oct 30 21:26:32.091969 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.091945    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7b0562c-6755-48c3-a8df-c26a16ad4c22" path="/var/lib/kubelet/pods/e7b0562c-6755-48c3-a8df-c26a16ad4c22/volumes"
Oct 30 21:26:32.250697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.250664    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:32.251546 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.251523    4852 patch_prober.go:29] interesting pod/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 21:26:32.251670 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.251574    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 21:26:32.742943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.742910    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:32.872413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.871153    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5" event={"ID":"b69450b8-9338-46e2-9c5a-094fad128ea4","Type":"ContainerStarted","Data":"42a5a1f5f3a150542aa9c41decc938905593300ee0254c781a683a44ed0132c8"}
Oct 30 21:26:32.872413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.871410    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:32.907614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.907558    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5" podStartSLOduration=2.368246101 podStartE2EDuration="2.907516559s" totalImagesPullingTime="539.270458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:32.906314192 +0000 UTC m=+2901.021048397" watchObservedRunningTime="2026-10-30 21:26:32.907516559 +0000 UTC m=+2901.022250741"
Oct 30 21:26:32.908868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:32.908833    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podStartSLOduration=2.90882361 podStartE2EDuration="2.90882361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:31.832358893 +0000 UTC m=+2899.947093084" watchObservedRunningTime="2026-10-30 21:26:32.90882361 +0000 UTC m=+2901.023557799"
Oct 30 21:26:33.262426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:33.261506    4852 patch_prober.go:29] interesting pod/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 21:26:33.262426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:33.261581    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 21:26:33.542200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:33.542111    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:34.229652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:34.229617    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:34.262411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:34.258963    4852 patch_prober.go:29] interesting pod/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 21:26:34.262411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:34.259537    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 21:26:35.113610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:35.113571    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:35.258410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:35.255699    4852 patch_prober.go:29] interesting pod/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 21:26:35.258410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:35.255759    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 21:26:35.832008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:35.831964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.249278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:36.249199    4852 patch_prober.go:29] interesting pod/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 21:26:36.249278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:36.249256    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 21:26:36.526607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:36.526573    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.995956 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:36.995317    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.142721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:37.142685    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.259240 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:37.259207    4852 patch_prober.go:29] interesting pod/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 21:26:37.259578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:37.259548    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 21:26:37.323816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:37.323774    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.492566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:37.492537    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.620148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:37.619978    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:38.361271 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:38.361235    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:38.387933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:38.387901    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:38.389448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:38.389417    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:38.390146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:38.390122    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:26:38.897009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:38.896967    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:39.120528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:39.120488    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:39.834876 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:39.834846    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:39.944906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:39.944872    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:26:40.634693 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:40.634664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:41.368301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:41.368251    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:42.029800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:42.029766    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:42.797628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:42.797539    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:43.542554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:43.542517    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:44.311596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:44.311558    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.126493 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:45.126454    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.634088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:45.634053    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.735340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:45.735306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.974855 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:45.974769    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:46.736434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:46.730844    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:47.854476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:47.854436    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:48.877209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:48.877176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:49.832836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:49.832796    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:50.646599 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:50.646570    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:51.507923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:51.507885    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:51.606965 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:26:51.606923    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:26:52.321821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:52.321779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:53.011871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:53.011825    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:53.203310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:53.203264    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:53.666048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:53.666013    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:53.746312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:53.746273    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:54.125100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:54.125064    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:54.509937 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:54.509904    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:55.393974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:55.393940    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:56.123343 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:56.123309    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:56.854817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:56.854782    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:57.139448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:57.139337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:57.585595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:57.585558    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:57.729671 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:57.729633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.049584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:58.049545    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.231911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:58.231861    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.726768 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:58.726727    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:59.401110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:26:59.401072    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:00.208942 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:00.208902    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.118777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:01.118741    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.505758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:01.505716    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.606976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:01.606902    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.833656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:01.833575    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"]
Oct 30 21:27:01.833849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:01.833824    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy" containerID="cri-o://95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4" gracePeriod=30
Oct 30 21:27:01.848236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:01.848202    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.100188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:02.082232    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.261205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:02.261092    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.370590 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:02.370463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.471274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:02.471231    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:03.311333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.311297    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:03.639182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.639071    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:03.740301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.740265    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:03.768351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.768315    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"]
Oct 30 21:27:03.773200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.773169    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:03.782505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.782478    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"]
Oct 30 21:27:03.855412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.846044    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-f8e2c3-dockercfg-l495n\""
Oct 30 21:27:03.855412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:03.846137    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-f8e2c3\""
Oct 30 21:27:04.043035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.042997    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f059960-bfce-4cda-bf8c-e4be7abf437f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:04.043244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.043086    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb55m\" (UniqueName: \"kubernetes.io/projected/0f059960-bfce-4cda-bf8c-e4be7abf437f-kube-api-access-zb55m\") pod \"payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:04.145161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.145116    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f059960-bfce-4cda-bf8c-e4be7abf437f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:04.145381 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.145194    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb55m\" (UniqueName: \"kubernetes.io/projected/0f059960-bfce-4cda-bf8c-e4be7abf437f-kube-api-access-zb55m\") pod \"payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:04.145882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.145861    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f059960-bfce-4cda-bf8c-e4be7abf437f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:04.154146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.154120    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb55m\" (UniqueName: \"kubernetes.io/projected/0f059960-bfce-4cda-bf8c-e4be7abf437f-kube-api-access-zb55m\") pod \"payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:04.394548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.394454    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:04.446145 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.446105    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:04.682478 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:27:04.682444    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f059960_bfce_4cda_bf8c_e4be7abf437f.slice/crio-182f94addfa728b6528a6bcb52947b6ead8115c1c7226670c63ae9bc400d4bac WatchSource:0}: Error finding container 182f94addfa728b6528a6bcb52947b6ead8115c1c7226670c63ae9bc400d4bac: Status 404 returned error can't find the container with id 182f94addfa728b6528a6bcb52947b6ead8115c1c7226670c63ae9bc400d4bac
Oct 30 21:27:04.683036 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:04.682973    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"]
Oct 30 21:27:05.350483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:05.350439    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw" event={"ID":"0f059960-bfce-4cda-bf8c-e4be7abf437f","Type":"ContainerStarted","Data":"97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9"}
Oct 30 21:27:05.350690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:05.350545    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw" event={"ID":"0f059960-bfce-4cda-bf8c-e4be7abf437f","Type":"ContainerStarted","Data":"182f94addfa728b6528a6bcb52947b6ead8115c1c7226670c63ae9bc400d4bac"}
Oct 30 21:27:05.350690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:05.350578    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:05.371313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:05.371264    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw" podStartSLOduration=2.371252499 podStartE2EDuration="2.371252499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:05.365891521 +0000 UTC m=+2933.480625713" watchObservedRunningTime="2026-10-30 21:27:05.371252499 +0000 UTC m=+2933.485986760"
Oct 30 21:27:05.413257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:05.413231    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:05.539501 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:05.527552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:06.249781 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:06.249747    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:07.092923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.092887    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:07.120218 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.120144    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 21:27:07.195861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.195822    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-credential-socket\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196054 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.195980    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-kube-api-access-npdpd\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-kube-api-access-npdpd\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196052    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-certs\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196113    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-credential-socket" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:07.196258 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196144    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-socket\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196308    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-data\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196364    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-token\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-token\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196438    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istiod-ca-cert\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196493    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-envoy\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196541    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-podinfo\") pod \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\" (UID: \"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da\") "
Oct 30 21:27:07.196710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196690    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-socket" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:07.196868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.196849    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-certs" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:07.197228 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.197207    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.197310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.197233    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.197310 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.197249    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.197901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.197871    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-data" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:07.198276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.198251    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istiod-ca-cert" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:27:07.200708 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.200669    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-token" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:07.201800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.201773    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-envoy" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:07.211087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.208808    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-kube-api-access-npdpd" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "kube-api-access-npdpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:07.211087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.208846    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-podinfo" pod "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" (UID: "e2419084-bb6e-4dea-b50a-6d5b4f5ab0da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:27:07.298347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.298305    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.298347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.298345    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.298347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.298355    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.298674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.298364    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.298674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.298373    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.298674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.298381    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npdpd\" (UniqueName: \"kubernetes.io/projected/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da-kube-api-access-npdpd\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:07.371376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.371275    4852 generic.go:356] "Generic (PLEG): container finished" podID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerID="95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4" exitCode=0
Oct 30 21:27:07.371376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.371329    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"
Oct 30 21:27:07.371661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.371423    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" event={"ID":"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da","Type":"ContainerDied","Data":"95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4"}
Oct 30 21:27:07.371661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.371508    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh" event={"ID":"e2419084-bb6e-4dea-b50a-6d5b4f5ab0da","Type":"ContainerDied","Data":"870b37a5e6cc2ba837c4e109106835c1bc660de1997030b561bf8177e0689cf8"}
Oct 30 21:27:07.371661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.371554    4852 scope.go:122] "RemoveContainer" containerID="95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4"
Oct 30 21:27:07.389281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.389261    4852 scope.go:122] "RemoveContainer" containerID="95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4"
Oct 30 21:27:07.389648 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:27:07.389625    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4\": container with ID starting with 95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4 not found: ID does not exist" containerID="95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4"
Oct 30 21:27:07.389710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.389660    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4"} err="failed to get container status \"95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4\": rpc error: code = NotFound desc = could not find container \"95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4\": container with ID starting with 95161be2e56e3a28051a22eea9abcb27232b663f507e6f40d2b62a19db4b18f4 not found: ID does not exist"
Oct 30 21:27:07.402538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.402508    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"]
Oct 30 21:27:07.406790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.406763    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-8e4acb-openshift-default-868dc48fff-5pbvh"]
Oct 30 21:27:07.773365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:07.773328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:08.006849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:08.006818    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" path="/var/lib/kubelet/pods/e2419084-bb6e-4dea-b50a-6d5b4f5ab0da/volumes"
Oct 30 21:27:08.476038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:08.476001    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:09.236366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:09.236331    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:09.931128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:09.931091    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:10.879455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:10.879377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:11.703422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:11.703355    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:12.427269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:12.427233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:13.238888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:13.238852    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:14.011538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:14.011507    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:14.724572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:14.724539    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:15.411868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:15.411827    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:16.137506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:16.137461    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:16.363366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:16.363336    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:27:16.902971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:16.902931    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:17.602911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:17.602875    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:18.325570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:18.325530    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:19.108171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:19.108129    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:19.800618 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:19.800582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:20.531744 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:20.531708    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:21.221931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:21.221893    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:21.559267 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:27:21.559230    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:27:21.950656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:21.950550    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:22.085943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:22.085906    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:22.182559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:22.182522    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:22.932312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:22.932281    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:23.708371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:23.708325    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:23.997252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:23.997154    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:24.433232 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:24.433182    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:25.239313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:25.239276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:26.127459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:26.127362    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:26.839178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:26.839143    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:27.656283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:27.656246    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:28.417398 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:28.417353    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:29.230038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:29.230001    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:30.127513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:30.127474    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:30.930420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:30.930358    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:31.700818 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:31.700786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:32.411817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:32.411779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:33.237270 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:33.237230    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:33.993400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:33.993361    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:34.682072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:34.682025    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:35.428267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:35.428226    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:36.110906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:36.110870    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:36.936993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:36.936953    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:37.637211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:37.637177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:38.322246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:38.322207    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:39.009519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:39.009485    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:39.730549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:39.730512    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:40.510689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:40.510650    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:41.405009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:41.404978    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:42.236650 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:42.236604    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:43.032869 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:43.032826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:44.133722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:44.133672    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:45.092057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:45.092012    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:45.852565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:45.852519    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:46.740902 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:46.740868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.628215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:47.628177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.740409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:47.736443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:48.371597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:48.371561    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:49.136540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:49.136502    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:49.823203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:49.823148    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:50.561565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:50.561528    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:51.319899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:51.319862    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:51.419925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:51.419893    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:51.606692 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:27:51.606581    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:27:52.211351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:52.211321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:52.930371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:52.930332    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:53.935583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:53.935554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:54.715880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:54.715842    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:55.554340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:55.554307    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:56.327865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:56.327827    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:57.219827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:57.219765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:58.019257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:58.019223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:59.074513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:27:59.074477    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:00.324987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:00.324947    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:01.140804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:01.140767    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:01.931063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:01.931009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:02.670151 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:02.670113    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:02.769025 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:02.768995    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:03.516637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:03.516601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:04.219031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:04.218994    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:05.018882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:05.018850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:05.492456 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:05.492340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:05.836219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:05.836186    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:06.600284 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:06.600248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:07.309481 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:07.309442    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:08.046509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:08.046455    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:09.021334 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:09.021301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:09.808490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:09.808451    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:10.531552 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:10.531512    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:11.305814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:11.305780    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:12.010758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:12.010730    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:12.733943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:12.733867    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:13.522806 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:13.522774    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:14.215129 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:14.215093    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:14.941714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:14.941680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:15.721330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:15.721298    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:16.455173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:16.455124    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:17.245168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:17.245136    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:17.993732 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:17.993701    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:18.710518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:18.710478    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:19.512322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:19.512292    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:20.642460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:20.642412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:21.559570 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:28:21.559524    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:28:21.626362 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:21.626322    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:22.366667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:22.366634    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:23.127236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:23.127203    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:23.931854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:23.931820    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:24.609491 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:24.609325    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:25.306934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:25.306900    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:26.129364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:26.129306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:26.898852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:26.898806    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:27.595294 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:27.595223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:28.333957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:28.333925    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:29.111435 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:29.111366    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:29.832983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:29.832939    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:30.655328 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:30.655295    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:31.442266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:31.442236    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:32.311139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:32.311102    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:33.065457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:33.065424    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:33.929613 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:33.929581    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:34.604883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:34.604841    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:35.442760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:35.442708    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:36.206778 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:36.206744    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:36.958345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:36.958306    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:37.731373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:37.731334    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:38.518575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:38.518540    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:39.236772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:39.236734    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:39.928899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:39.928866    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:40.629348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:40.629309    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:41.504752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:41.504720    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:42.219457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:42.219382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:43.007458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.007425    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:43.166171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.166133    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"]
Oct 30 21:28:43.166911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.166892    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy"
Oct 30 21:28:43.166971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.166912    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy"
Oct 30 21:28:43.167074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.167064    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2419084-bb6e-4dea-b50a-6d5b4f5ab0da" containerName="istio-proxy"
Oct 30 21:28:43.170971 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.170953    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.176215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.176186    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f8e2c3-openshift-default-dockercfg-b259x\""
Oct 30 21:28:43.185309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.185264    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"]
Oct 30 21:28:43.280628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.280535    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-envoy\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.280628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.280575    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-credential-socket\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.280628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.280605    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-socket\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.281092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.280790    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-certs\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.281092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.280891    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-podinfo\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.281092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.280930    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-token\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.281092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.280998    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cpdl\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-kube-api-access-5cpdl\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.281092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.281046    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d867c0a9-5a70-4024-8481-35f6f5c545db-istiod-ca-cert\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.281092 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.281074    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-data\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.382970    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-certs\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383058    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-podinfo\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383101    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-token\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383155    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cpdl\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-kube-api-access-5cpdl\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383217    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d867c0a9-5a70-4024-8481-35f6f5c545db-istiod-ca-cert\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383254    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-data\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383428    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-envoy\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383455    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-certs\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383461    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-credential-socket\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383550    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-socket\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383768    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-data\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383911 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383857    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-credential-socket\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.383957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.383935    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-socket\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.384027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.384005    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d867c0a9-5a70-4024-8481-35f6f5c545db-istiod-ca-cert\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.385733 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.385711    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-envoy\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.385840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.385772    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-podinfo\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.391408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.391368    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-token\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.391556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.391538    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cpdl\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-kube-api-access-5cpdl\") pod \"e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") " pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.491796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.491753    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:43.792139 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:28:43.792089    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd867c0a9_5a70_4024_8481_35f6f5c545db.slice/crio-298ae3734ecacc68301433e8ce757dccd0a2b899bded8eb83af5fbca17db4355 WatchSource:0}: Error finding container 298ae3734ecacc68301433e8ce757dccd0a2b899bded8eb83af5fbca17db4355: Status 404 returned error can't find the container with id 298ae3734ecacc68301433e8ce757dccd0a2b899bded8eb83af5fbca17db4355
Oct 30 21:28:43.792870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.792842    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"]
Oct 30 21:28:43.795081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.795047    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:28:43.795169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.795116    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:28:43.795169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.795157    4852 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 30 21:28:43.805409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:43.805320    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:44.445311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:44.445270    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" event={"ID":"d867c0a9-5a70-4024-8481-35f6f5c545db","Type":"ContainerStarted","Data":"62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359"}
Oct 30 21:28:44.445759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:44.445331    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" event={"ID":"d867c0a9-5a70-4024-8481-35f6f5c545db","Type":"ContainerStarted","Data":"298ae3734ecacc68301433e8ce757dccd0a2b899bded8eb83af5fbca17db4355"}
Oct 30 21:28:44.467331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:44.467274    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podStartSLOduration=1.467259649 podStartE2EDuration="1.467259649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:44.463205493 +0000 UTC m=+3032.577939684" watchObservedRunningTime="2026-10-30 21:28:44.467259649 +0000 UTC m=+3032.581993838"
Oct 30 21:28:44.492273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:44.492238    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:44.494212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:44.494177    4852 patch_prober.go:29] interesting pod/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused" start-of-body=
Oct 30 21:28:44.494402 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:44.494232    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused"
Oct 30 21:28:44.543894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:44.543853    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:45.231453 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:45.231420    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:45.492345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:45.492255    4852 patch_prober.go:29] interesting pod/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused" start-of-body=
Oct 30 21:28:45.492345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:45.492310    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused"
Oct 30 21:28:45.902472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:45.902439    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:46.492705 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:46.492671    4852 patch_prober.go:29] interesting pod/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused" start-of-body=
Oct 30 21:28:46.493122 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:46.492736    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused"
Oct 30 21:28:46.738125 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:46.738094    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:47.492175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:47.492139    4852 patch_prober.go:29] interesting pod/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused" start-of-body=
Oct 30 21:28:47.492367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:47.492200    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused"
Oct 30 21:28:47.509018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:47.508968    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:48.323057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:48.323020    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:48.492802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:48.492763    4852 patch_prober.go:29] interesting pod/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused" start-of-body=
Oct 30 21:28:48.492986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:48.492835    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused"
Oct 30 21:28:49.147440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:49.147410    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:49.492910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:49.492823    4852 patch_prober.go:29] interesting pod/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused" start-of-body=
Oct 30 21:28:49.492910 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:49.492880    4852 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.185:15021/healthz/ready\": dial tcp 10.133.0.185:15021: connect: connection refused"
Oct 30 21:28:49.909055 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:49.909011    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:50.559983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:50.559945    4852 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:50.560715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:50.560676    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:50.562088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:50.562070    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:28:50.723782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:50.723740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:51.567645 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:28:51.567601    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:28:51.654740 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:51.654608    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:52.429057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:52.429019    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:53.245827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:53.245786    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:54.008694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:54.008641    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:54.709448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:54.709370    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:55.410381 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:55.410321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:56.142870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:56.142832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:56.934220 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:56.934183    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:57.629486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:57.629449    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:58.447625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:58.447594    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:59.241844 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:59.241811    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:59.947340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:28:59.947286    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:00.714627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:00.714556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:01.713447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:01.713382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:02.404660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:02.404623    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:03.219634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:03.219537    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:03.936119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:03.936078    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:04.621341 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:04.621314    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:05.352512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:05.352479    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:05.544769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:05.544726    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:06.126995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:06.126946    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:06.832018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:06.831983    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:07.544766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:07.544728    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:08.320802 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:08.320768    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:09.219728 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:09.219691    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:10.039598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:10.039563    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:11.024897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:11.024865    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:11.800892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:11.800848    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:12.752757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:12.749635    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:13.536925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:13.536884    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:14.357687 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:14.357648    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:15.347824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:15.347793    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:16.187462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:16.187425    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:17.007768 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:17.007726    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:17.810198 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:17.810158    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:18.765227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:18.765195    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:19.434791 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:19.434751    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:20.304952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:20.304913    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:21.047790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:21.047757    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:21.559664 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:29:21.559628    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:29:21.847351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:21.847259    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:22.570947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:22.570910    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:23.255957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:23.255925    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:24.015661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:24.015629    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:24.721702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:24.721635    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:25.414523 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:25.414482    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:25.905642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:25.905606    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:26.385076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:26.384989    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:27.033717 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:27.033671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:27.732675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:27.732631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:28.491450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:28.491408    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:29.422051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:29.422008    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:30.385821 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:30.385772    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:31.162360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:31.162316    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:31.920980 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:31.920943    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:32.687494 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:32.687458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:33.536673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:33.536631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:34.416329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:34.416293    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:35.180182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:35.180087    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:36.012457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:36.012423    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:36.740078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:36.740035    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:37.026291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:37.026249    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:37.981694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:37.981664    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:38.833566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:38.833530    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:39.694164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:39.694130    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:40.395429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:40.395382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:41.323760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:41.323723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:42.120736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:42.120693    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:42.826789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:42.826695    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:43.512272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:43.512242    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:44.331465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:44.331351    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:44.450782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:44.450747    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:45.201420 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:45.201357    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:46.018562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:46.018529    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:46.834331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:46.834298    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:47.511017 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:47.510946    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:47.617363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:47.617319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:47.717156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:47.717120    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:48.470835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:48.470798    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:49.229252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:49.229221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:50.012458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:50.012421    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:50.833187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:50.833150    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:51.559886 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:29:51.559846    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:29:51.637186 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:51.637146    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:52.422556 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:52.422524    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:53.216875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:53.216843    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:54.040463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:54.040428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:54.814592 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:54.814556    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:55.360581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:55.360553    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:55.615734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:55.615653    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:56.062583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:56.062547    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"]
Oct 30 21:29:56.062816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:56.062789    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy" containerID="cri-o://b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107" gracePeriod=30
Oct 30 21:29:56.067995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:56.067964    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"]
Oct 30 21:29:56.068221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:56.068190    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy" containerID="cri-o://a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f" gracePeriod=30
Oct 30 21:29:56.075224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:56.075188    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"]
Oct 30 21:29:56.075561 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:56.075528    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy" containerID="cri-o://ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca" gracePeriod=30
Oct 30 21:29:56.628707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:56.628657    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:57.422157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:57.422125    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:58.127372 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:58.127335    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:58.939625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:58.939588    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:59.717895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:29:59.717858    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:00.180519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.180477    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"]
Oct 30 21:30:00.186359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.186326    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"]
Oct 30 21:30:00.187644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.187622    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.193081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.193052    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.199630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.199604    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"]
Oct 30 21:30:00.235256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.235215    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"]
Oct 30 21:30:00.244207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.244067    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"]
Oct 30 21:30:00.253634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.253590    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.257677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.257479    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"]
Oct 30 21:30:00.272063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.272037    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.275657 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.273985    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 21:30:00.277539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.277512    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"]
Oct 30 21:30:00.290633 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.290606    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"]
Oct 30 21:30:00.299424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.298702    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"]
Oct 30 21:30:00.304300 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.304273    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.308698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.308670    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"]
Oct 30 21:30:00.315352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.314622    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"]
Oct 30 21:30:00.320431 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.320366    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.326948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.326922    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"]
Oct 30 21:30:00.328104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.328066    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8sk9\" (UniqueName: \"kubernetes.io/projected/0066ff78-850a-4603-8eaa-0b533e03f28d-kube-api-access-q8sk9\") pod \"maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.328514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.328490    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ff78-850a-4603-8eaa-0b533e03f28d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.328732 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.328712    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7944\" (UniqueName: \"kubernetes.io/projected/40028a96-aedb-4c5a-9651-f1784ba3f52f-kube-api-access-f7944\") pod \"maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.329652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.329629    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40028a96-aedb-4c5a-9651-f1784ba3f52f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.334896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.332685    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.338178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.337914    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"]
Oct 30 21:30:00.351060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.349311    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"]
Oct 30 21:30:00.382342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.382303    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"]
Oct 30 21:30:00.389248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.389216    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.429163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.429130    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"]
Oct 30 21:30:00.431207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431136    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7944\" (UniqueName: \"kubernetes.io/projected/40028a96-aedb-4c5a-9651-f1784ba3f52f-kube-api-access-f7944\") pod \"maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.431344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431218    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67582\" (UniqueName: \"kubernetes.io/projected/cec79507-2620-4011-9ee9-04547f6810de-kube-api-access-67582\") pod \"maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.431344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431252    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c57b1-f837-4b73-8862-b345dd150076-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.431344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431278    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99126667-d257-4026-8b8c-82c227692301-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.431344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431324    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40028a96-aedb-4c5a-9651-f1784ba3f52f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.431539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431362    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec79507-2620-4011-9ee9-04547f6810de-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.431539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431481    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwvfc\" (UniqueName: \"kubernetes.io/projected/99126667-d257-4026-8b8c-82c227692301-kube-api-access-wwvfc\") pod \"maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.431652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431632    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8sk9\" (UniqueName: \"kubernetes.io/projected/0066ff78-850a-4603-8eaa-0b533e03f28d-kube-api-access-q8sk9\") pod \"maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.431718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431675    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bcee80-0a44-4c57-bc04-19616abc74ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.431781 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431761    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921d6df-789f-4dca-8a1e-14c3d443fbc7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.431824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431791    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vrzr\" (UniqueName: \"kubernetes.io/projected/f921d6df-789f-4dca-8a1e-14c3d443fbc7-kube-api-access-2vrzr\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.431864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431850    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ff78-850a-4603-8eaa-0b533e03f28d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.431927 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431905    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfqns\" (UniqueName: \"kubernetes.io/projected/57bcee80-0a44-4c57-bc04-19616abc74ff-kube-api-access-pfqns\") pod \"maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.432003 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.431941    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l87m\" (UniqueName: \"kubernetes.io/projected/2d5c57b1-f837-4b73-8862-b345dd150076-kube-api-access-7l87m\") pod \"maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.433101 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.433074    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ff78-850a-4603-8eaa-0b533e03f28d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.433260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.433112    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40028a96-aedb-4c5a-9651-f1784ba3f52f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.439082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.439044    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"]
Oct 30 21:30:00.439235 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.439207    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.448496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.448465    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"]
Oct 30 21:30:00.457801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.457769    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8sk9\" (UniqueName: \"kubernetes.io/projected/0066ff78-850a-4603-8eaa-0b533e03f28d-kube-api-access-q8sk9\") pod \"maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.457952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.457769    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7944\" (UniqueName: \"kubernetes.io/projected/40028a96-aedb-4c5a-9651-f1784ba3f52f-kube-api-access-f7944\") pod \"maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.524713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.524676    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:00.533814 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.533785    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwvfc\" (UniqueName: \"kubernetes.io/projected/99126667-d257-4026-8b8c-82c227692301-kube-api-access-wwvfc\") pod \"maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.533954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.533889    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/323e2d2f-adce-4489-81f6-159d46f03848-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.534001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.533965    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bcee80-0a44-4c57-bc04-19616abc74ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.534048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534031    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkg74\" (UniqueName: \"kubernetes.io/projected/323e2d2f-adce-4489-81f6-159d46f03848-kube-api-access-gkg74\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.534157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534141    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921d6df-789f-4dca-8a1e-14c3d443fbc7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.534224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534171    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vrzr\" (UniqueName: \"kubernetes.io/projected/f921d6df-789f-4dca-8a1e-14c3d443fbc7-kube-api-access-2vrzr\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.534224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534202    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m62rr\" (UniqueName: \"kubernetes.io/projected/eebf6d42-2bac-44f1-bbd5-f7bc24981893-kube-api-access-m62rr\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.534824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534280    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfqns\" (UniqueName: \"kubernetes.io/projected/57bcee80-0a44-4c57-bc04-19616abc74ff-kube-api-access-pfqns\") pod \"maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.534824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534316    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l87m\" (UniqueName: \"kubernetes.io/projected/2d5c57b1-f837-4b73-8862-b345dd150076-kube-api-access-7l87m\") pod \"maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.534824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534448    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebf6d42-2bac-44f1-bbd5-f7bc24981893-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.534824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534543    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67582\" (UniqueName: \"kubernetes.io/projected/cec79507-2620-4011-9ee9-04547f6810de-kube-api-access-67582\") pod \"maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.534824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534599    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c57b1-f837-4b73-8862-b345dd150076-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.534824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534639    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99126667-d257-4026-8b8c-82c227692301-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.534824 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534712    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec79507-2620-4011-9ee9-04547f6810de-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.535105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534831    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bcee80-0a44-4c57-bc04-19616abc74ff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.535105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.534930    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921d6df-789f-4dca-8a1e-14c3d443fbc7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.535497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.535476    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec79507-2620-4011-9ee9-04547f6810de-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.535600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.535478    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c57b1-f837-4b73-8862-b345dd150076-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.535600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.535537    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99126667-d257-4026-8b8c-82c227692301-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.542407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.542375    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:00.544871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.544847    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfqns\" (UniqueName: \"kubernetes.io/projected/57bcee80-0a44-4c57-bc04-19616abc74ff-kube-api-access-pfqns\") pod \"maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.557734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.548724    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l87m\" (UniqueName: \"kubernetes.io/projected/2d5c57b1-f837-4b73-8862-b345dd150076-kube-api-access-7l87m\") pod \"maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.557734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.548996    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vrzr\" (UniqueName: \"kubernetes.io/projected/f921d6df-789f-4dca-8a1e-14c3d443fbc7-kube-api-access-2vrzr\") pod \"maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.557734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.549600    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67582\" (UniqueName: \"kubernetes.io/projected/cec79507-2620-4011-9ee9-04547f6810de-kube-api-access-67582\") pod \"maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.557734 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.550290    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwvfc\" (UniqueName: \"kubernetes.io/projected/99126667-d257-4026-8b8c-82c227692301-kube-api-access-wwvfc\") pod \"maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.580730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.580695    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:00.595237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.594961    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:00.641775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.641735    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m62rr\" (UniqueName: \"kubernetes.io/projected/eebf6d42-2bac-44f1-bbd5-f7bc24981893-kube-api-access-m62rr\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.641925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.641855    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebf6d42-2bac-44f1-bbd5-f7bc24981893-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.642144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.642114    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/323e2d2f-adce-4489-81f6-159d46f03848-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.642214 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.642188    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkg74\" (UniqueName: \"kubernetes.io/projected/323e2d2f-adce-4489-81f6-159d46f03848-kube-api-access-gkg74\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.644142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.644118    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:00.644503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.644447    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebf6d42-2bac-44f1-bbd5-f7bc24981893-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.645337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.645291    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/323e2d2f-adce-4489-81f6-159d46f03848-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.651630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.647030    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:00.665314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.664679    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m62rr\" (UniqueName: \"kubernetes.io/projected/eebf6d42-2bac-44f1-bbd5-f7bc24981893-kube-api-access-m62rr\") pod \"maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.674673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.673767    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkg74\" (UniqueName: \"kubernetes.io/projected/323e2d2f-adce-4489-81f6-159d46f03848-kube-api-access-gkg74\") pod \"maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.674673 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.674026    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:00.935647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.721709    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:00.935647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.824630    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:00.977082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:00.975295    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"]
Oct 30 21:30:01.038425 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:01.038375    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0066ff78_850a_4603_8eaa_0b533e03f28d.slice/crio-5706e478c0ecfb6c349de4ecda4741b592411fb9f8400a65d238ae8b2db9c28f WatchSource:0}: Error finding container 5706e478c0ecfb6c349de4ecda4741b592411fb9f8400a65d238ae8b2db9c28f: Status 404 returned error can't find the container with id 5706e478c0ecfb6c349de4ecda4741b592411fb9f8400a65d238ae8b2db9c28f
Oct 30 21:30:01.094715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.094653    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"]
Oct 30 21:30:01.102038 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.101999    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:01.309408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.306891    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"]
Oct 30 21:30:01.368436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.364761    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerStarted","Data":"103c28a439bfc3117cd1ad4346435aff49001411781706c7dcb4c6f80cc836ee"}
Oct 30 21:30:01.372597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.372542    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerStarted","Data":"5706e478c0ecfb6c349de4ecda4741b592411fb9f8400a65d238ae8b2db9c28f"}
Oct 30 21:30:01.375106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.375050    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerStarted","Data":"3d5d01604124ac8ce5c044ef77c7330f45d5fca03684c51ddb0495ca7777db5b"}
Oct 30 21:30:01.381300 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.381271    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"]
Oct 30 21:30:01.766158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.764854    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"]
Oct 30 21:30:01.784005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.783967    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"]
Oct 30 21:30:01.977737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:01.977694    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 21:30:02.044722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.044648    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"]
Oct 30 21:30:02.055057 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:02.055009    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323e2d2f_adce_4489_81f6_159d46f03848.slice/crio-015828d30c588f8bcea66753cd94f822d03b7a85ccb7b657f62b5e62c0c2b504 WatchSource:0}: Error finding container 015828d30c588f8bcea66753cd94f822d03b7a85ccb7b657f62b5e62c0c2b504: Status 404 returned error can't find the container with id 015828d30c588f8bcea66753cd94f822d03b7a85ccb7b657f62b5e62c0c2b504
Oct 30 21:30:02.065737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.065712    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"]
Oct 30 21:30:02.087537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.077224    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"]
Oct 30 21:30:02.096532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.091647    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:30:02.096532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.093333    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:30:02.110649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.110614    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-credential-socket\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.110846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.110701    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81e9db27-0286-4be5-aa15-22c15b7183cf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81e9db27-0286-4be5-aa15-22c15b7183cf-istiod-ca-cert\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.110918 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.110854    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-token\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-token\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.110980 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.110926    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-credential-socket" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.111050 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.110999    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-data\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.111050 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111023    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-certs\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.111172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111077    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-envoy\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.111172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111113    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-socket\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.111345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111188    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-podinfo\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.111345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111205    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81e9db27-0286-4be5-aa15-22c15b7183cf-istiod-ca-cert" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:02.111345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111235    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-kube-api-access-zz8lk\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-kube-api-access-zz8lk\") pod \"81e9db27-0286-4be5-aa15-22c15b7183cf\" (UID: \"81e9db27-0286-4be5-aa15-22c15b7183cf\") "
Oct 30 21:30:02.111532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111436    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-certs" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.111589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111541    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-data" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.111651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111631    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-socket" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.111964 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111938    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.112141 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111966    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.112141 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111982    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.112141 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.111995    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.112141 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.112010    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81e9db27-0286-4be5-aa15-22c15b7183cf-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.132316 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.132278    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:02.152058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.151657    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-token" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:02.153075 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:02.153033    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5c57b1_f837_4b73_8862_b345dd150076.slice/crio-386b55e2afef16b6f4c82bcc57c5f3153f8384fdb4c7e239f2b51916cbb0621b WatchSource:0}: Error finding container 386b55e2afef16b6f4c82bcc57c5f3153f8384fdb4c7e239f2b51916cbb0621b: Status 404 returned error can't find the container with id 386b55e2afef16b6f4c82bcc57c5f3153f8384fdb4c7e239f2b51916cbb0621b
Oct 30 21:30:02.157772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.157732    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-envoy" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.161555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.161521    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-podinfo" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:02.161960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.161932    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-kube-api-access-zz8lk" pod "81e9db27-0286-4be5-aa15-22c15b7183cf" (UID: "81e9db27-0286-4be5-aa15-22c15b7183cf"). InnerVolumeSpecName "kube-api-access-zz8lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:02.162757 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:02.162730    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebf6d42_2bac_44f1_bbd5_f7bc24981893.slice/crio-188410df43da0ccb88717375c74ca1fa6f50dde7143b3354cbd7a47ade1e33d4 WatchSource:0}: Error finding container 188410df43da0ccb88717375c74ca1fa6f50dde7143b3354cbd7a47ade1e33d4: Status 404 returned error can't find the container with id 188410df43da0ccb88717375c74ca1fa6f50dde7143b3354cbd7a47ade1e33d4
Oct 30 21:30:02.213042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.212931    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-envoy\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.213042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.212987    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istiod-ca-cert\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.213042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213040    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-token\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-token\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.213361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213094    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-socket\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213162    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-data\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.213361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213199    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-token\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-token\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213227    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istiod-ca-cert\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213255    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-certs\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.213361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213326    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-certs\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213358    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-kube-api-access-fjqbf\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-kube-api-access-fjqbf\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213437    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-credential-socket\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213461    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-socket" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213490    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-credential-socket\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213490    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istiod-ca-cert" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213519    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-data\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213606    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-envoy\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213689    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-data" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213706    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-kube-api-access-mbqs6\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-kube-api-access-mbqs6\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.213788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213762    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istiod-ca-cert" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:02.214237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213861    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-socket\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.214237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213904    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-podinfo\") pod \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\" (UID: \"90d3a5f7-4ba6-441f-a79c-cc326998db8e\") "
Oct 30 21:30:02.214237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213908    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-certs" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.214237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.213941    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-podinfo\") pod \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\" (UID: \"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e\") "
Oct 30 21:30:02.214237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.214109    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-data" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.214532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.214375    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-credential-socket" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.214532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.214438    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-credential-socket" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.214661 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.214565    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-socket" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.214887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.214866    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-certs" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.215532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215503    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215527    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215542    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215556    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz8lk\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-kube-api-access-zz8lk\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215569    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215583    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215595    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215608    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215620    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215635    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81e9db27-0286-4be5-aa15-22c15b7183cf-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215649    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215662    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215675    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.215696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.215692    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.251754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.251697    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-kube-api-access-fjqbf" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "kube-api-access-fjqbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:02.252166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.251888    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-token" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:02.301039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.273992    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-podinfo" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:02.301039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.291681    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-kube-api-access-mbqs6" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "kube-api-access-mbqs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:02.301039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.293726    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-token" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:02.301039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.294020    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-envoy" pod "90d3a5f7-4ba6-441f-a79c-cc326998db8e" (UID: "90d3a5f7-4ba6-441f-a79c-cc326998db8e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.301039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.294221    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-podinfo" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:02.301039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.296626    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-envoy" pod "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" (UID: "1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:02.318076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318041    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.318076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318077    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjqbf\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-kube-api-access-fjqbf\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.318318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318094    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.318318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318108    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbqs6\" (UniqueName: \"kubernetes.io/projected/90d3a5f7-4ba6-441f-a79c-cc326998db8e-kube-api-access-mbqs6\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.318318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318121    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d3a5f7-4ba6-441f-a79c-cc326998db8e-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.318318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318144    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.318318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318158    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.318318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.318171    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:02.385073 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.385031    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerStarted","Data":"f95c154e637302cfee4a9b27ea95a0596a5ced6ca6d20e4074064b57e49850c0"}
Oct 30 21:30:02.385241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.385092    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerStarted","Data":"981a0542345964643b5a72ba3f8dce29294803a45679bf3c819f78c17ce58059"}
Oct 30 21:30:02.386815 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.386783    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerStarted","Data":"d17715205f185723b7dad4aa54354719c54a640313af615a1df5e09eeee63137"}
Oct 30 21:30:02.388259 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.388235    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerStarted","Data":"188410df43da0ccb88717375c74ca1fa6f50dde7143b3354cbd7a47ade1e33d4"}
Oct 30 21:30:02.390164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.389534    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerStarted","Data":"09b6dbc724ab68df59d39b56773d97fb7dcf36207345770b1ed5be3c96c448b4"}
Oct 30 21:30:02.391238 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.391217    4852 generic.go:356] "Generic (PLEG): container finished" podID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerID="ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca" exitCode=0
Oct 30 21:30:02.391345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.391261    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"
Oct 30 21:30:02.391345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.391283    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" event={"ID":"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e","Type":"ContainerDied","Data":"ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca"}
Oct 30 21:30:02.391345 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.391321    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5" event={"ID":"1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e","Type":"ContainerDied","Data":"4b038dbc343c2b0e1b2b03d5aa5a9b700009a7d84daafe62051b9bc427ab785b"}
Oct 30 21:30:02.391542 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.391352    4852 scope.go:122] "RemoveContainer" containerID="ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca"
Oct 30 21:30:02.392779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.392757    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerStarted","Data":"015828d30c588f8bcea66753cd94f822d03b7a85ccb7b657f62b5e62c0c2b504"}
Oct 30 21:30:02.394707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.394675    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerStarted","Data":"4fbcc8b2ad419aa45655620627312ddd0317427d3d23509e3fb6a980bc08595b"}
Oct 30 21:30:02.396313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.396289    4852 generic.go:356] "Generic (PLEG): container finished" podID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerID="a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f" exitCode=0
Oct 30 21:30:02.396441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.396331    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"
Oct 30 21:30:02.396441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.396380    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" event={"ID":"81e9db27-0286-4be5-aa15-22c15b7183cf","Type":"ContainerDied","Data":"a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f"}
Oct 30 21:30:02.396549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.396447    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94" event={"ID":"81e9db27-0286-4be5-aa15-22c15b7183cf","Type":"ContainerDied","Data":"933c49e13db48b40bb8cd99964df58e36744c9db6578d05206e3b3f393087098"}
Oct 30 21:30:02.398442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.398418    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerStarted","Data":"f61952e4feaeab82d10494e8424d1ea2b0b3d7e71c0bd72ba4cc88ef3cd288fa"}
Oct 30 21:30:02.399836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.399814    4852 generic.go:356] "Generic (PLEG): container finished" podID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerID="b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107" exitCode=0
Oct 30 21:30:02.399932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.399864    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"
Oct 30 21:30:02.399932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.399889    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" event={"ID":"90d3a5f7-4ba6-441f-a79c-cc326998db8e","Type":"ContainerDied","Data":"b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107"}
Oct 30 21:30:02.399932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.399927    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22" event={"ID":"90d3a5f7-4ba6-441f-a79c-cc326998db8e","Type":"ContainerDied","Data":"15f750bd145b5d4c62fdf4441efaf4fbc258afdd5ee2c87d7342c083deda4c67"}
Oct 30 21:30:02.401801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.401660    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" event={"ID":"99126667-d257-4026-8b8c-82c227692301","Type":"ContainerStarted","Data":"3a1ca8be5690cf77f35248c11c56f68c59ea1c0bce6c1d5aaf074450a06d6db8"}
Oct 30 21:30:02.403229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.403206    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" event={"ID":"2d5c57b1-f837-4b73-8862-b345dd150076","Type":"ContainerStarted","Data":"386b55e2afef16b6f4c82bcc57c5f3153f8384fdb4c7e239f2b51916cbb0621b"}
Oct 30 21:30:02.406183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.406135    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" podStartSLOduration=2.406115978 podStartE2EDuration="2.406115978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:02.403905565 +0000 UTC m=+3110.518639756" watchObservedRunningTime="2026-10-30 21:30:02.406115978 +0000 UTC m=+3110.520850169"
Oct 30 21:30:02.430456 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.430340    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" podStartSLOduration=2.430323697 podStartE2EDuration="2.430323697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:02.427692555 +0000 UTC m=+3110.542426752" watchObservedRunningTime="2026-10-30 21:30:02.430323697 +0000 UTC m=+3110.545057892"
Oct 30 21:30:02.447694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.447657    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"]
Oct 30 21:30:02.457704 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.457672    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-xr9v5"]
Oct 30 21:30:02.469093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.469063    4852 scope.go:122] "RemoveContainer" containerID="ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca"
Oct 30 21:30:02.469483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.469438    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" podStartSLOduration=2.469425241 podStartE2EDuration="2.469425241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:02.46929593 +0000 UTC m=+3110.584030121" watchObservedRunningTime="2026-10-30 21:30:02.469425241 +0000 UTC m=+3110.584159431"
Oct 30 21:30:02.470527 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:02.470500    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca\": container with ID starting with ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca not found: ID does not exist" containerID="ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca"
Oct 30 21:30:02.470609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.470543    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca"} err="failed to get container status \"ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca\": rpc error: code = NotFound desc = could not find container \"ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca\": container with ID starting with ad775e39b03485633b35e00333891150bdf44ce31ce417e597ef016e592167ca not found: ID does not exist"
Oct 30 21:30:02.470609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.470572    4852 scope.go:122] "RemoveContainer" containerID="a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f"
Oct 30 21:30:02.492875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.492821    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" podStartSLOduration=2.492806101 podStartE2EDuration="2.492806101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:02.486878685 +0000 UTC m=+3110.601612875" watchObservedRunningTime="2026-10-30 21:30:02.492806101 +0000 UTC m=+3110.607540290"
Oct 30 21:30:02.505276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.505236    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"]
Oct 30 21:30:02.511994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.511958    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-x6r22"]
Oct 30 21:30:02.522246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.522191    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"]
Oct 30 21:30:02.526797 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.526767    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-77f556d647-6nj94"]
Oct 30 21:30:02.604510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.604338    4852 scope.go:122] "RemoveContainer" containerID="a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f"
Oct 30 21:30:02.604781 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:02.604749    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f\": container with ID starting with a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f not found: ID does not exist" containerID="a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f"
Oct 30 21:30:02.604896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.604794    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f"} err="failed to get container status \"a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f\": rpc error: code = NotFound desc = could not find container \"a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f\": container with ID starting with a89057976a3d6a724c54b7bb3b0e47a7ab806f7ee9e70f994180089f273d2b6f not found: ID does not exist"
Oct 30 21:30:02.604896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.604822    4852 scope.go:122] "RemoveContainer" containerID="b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107"
Oct 30 21:30:02.623438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.623409    4852 scope.go:122] "RemoveContainer" containerID="b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107"
Oct 30 21:30:02.623921 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:02.623893    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107\": container with ID starting with b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107 not found: ID does not exist" containerID="b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107"
Oct 30 21:30:02.624027 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.623933    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107"} err="failed to get container status \"b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107\": rpc error: code = NotFound desc = could not find container \"b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107\": container with ID starting with b8a66ee8cd74b6f4b90688b1d8dd64cacef58abcb9d6a12e3ac1f53c21496107 not found: ID does not exist"
Oct 30 21:30:02.907738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:02.907702    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:03.422422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.422300    4852 generic.go:356] "Generic (PLEG): container finished" podID="99126667-d257-4026-8b8c-82c227692301" containerID="73dfdce0eed9f3d5d900ae687a66c2b1ad64e44ce760123e92d27b57300aedfe" exitCode=60
Oct 30 21:30:03.423116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.423085    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" event={"ID":"99126667-d257-4026-8b8c-82c227692301","Type":"ContainerDied","Data":"73dfdce0eed9f3d5d900ae687a66c2b1ad64e44ce760123e92d27b57300aedfe"}
Oct 30 21:30:03.423736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.423716    4852 scope.go:122] "RemoveContainer" containerID="73dfdce0eed9f3d5d900ae687a66c2b1ad64e44ce760123e92d27b57300aedfe"
Oct 30 21:30:03.426065 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.426038    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" event={"ID":"2d5c57b1-f837-4b73-8862-b345dd150076","Type":"ContainerStarted","Data":"22f079279723ce8ed20a2dc5aab9b15971aab5b12b29d24fd68bbaa10d1aff5c"}
Oct 30 21:30:03.428507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.428479    4852 generic.go:356] "Generic (PLEG): container finished" podID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerID="f95c154e637302cfee4a9b27ea95a0596a5ced6ca6d20e4074064b57e49850c0" exitCode=60
Oct 30 21:30:03.428624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.428562    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerDied","Data":"f95c154e637302cfee4a9b27ea95a0596a5ced6ca6d20e4074064b57e49850c0"}
Oct 30 21:30:03.428779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.428763    4852 scope.go:122] "RemoveContainer" containerID="f95c154e637302cfee4a9b27ea95a0596a5ced6ca6d20e4074064b57e49850c0"
Oct 30 21:30:03.430982 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.430960    4852 generic.go:356] "Generic (PLEG): container finished" podID="cec79507-2620-4011-9ee9-04547f6810de" containerID="d17715205f185723b7dad4aa54354719c54a640313af615a1df5e09eeee63137" exitCode=60
Oct 30 21:30:03.431085 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.431025    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerDied","Data":"d17715205f185723b7dad4aa54354719c54a640313af615a1df5e09eeee63137"}
Oct 30 21:30:03.431318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.431298    4852 scope.go:122] "RemoveContainer" containerID="d17715205f185723b7dad4aa54354719c54a640313af615a1df5e09eeee63137"
Oct 30 21:30:03.432988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.432964    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerStarted","Data":"3e11f304bb75743e23ba82db8d5f91fe874f6d4fedb158c1b28c816953a30add"}
Oct 30 21:30:03.434732 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.434695    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerStarted","Data":"ee647b7aff5e8de3989fbda140400a162208080623814b8a3021ac1c53db9b23"}
Oct 30 21:30:03.438804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.438781    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerStarted","Data":"9e200f51c1f0e920611a510dca385bd0da7484b2ba3bbba0fd263beb1f30bb46"}
Oct 30 21:30:03.442719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.442691    4852 generic.go:356] "Generic (PLEG): container finished" podID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerID="4fbcc8b2ad419aa45655620627312ddd0317427d3d23509e3fb6a980bc08595b" exitCode=60
Oct 30 21:30:03.442835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.442776    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerDied","Data":"4fbcc8b2ad419aa45655620627312ddd0317427d3d23509e3fb6a980bc08595b"}
Oct 30 21:30:03.443266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.442950    4852 scope.go:122] "RemoveContainer" containerID="4fbcc8b2ad419aa45655620627312ddd0317427d3d23509e3fb6a980bc08595b"
Oct 30 21:30:03.446425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.446378    4852 generic.go:356] "Generic (PLEG): container finished" podID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerID="f61952e4feaeab82d10494e8424d1ea2b0b3d7e71c0bd72ba4cc88ef3cd288fa" exitCode=60
Oct 30 21:30:03.446552 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.446457    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerDied","Data":"f61952e4feaeab82d10494e8424d1ea2b0b3d7e71c0bd72ba4cc88ef3cd288fa"}
Oct 30 21:30:03.446706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.446689    4852 scope.go:122] "RemoveContainer" containerID="f61952e4feaeab82d10494e8424d1ea2b0b3d7e71c0bd72ba4cc88ef3cd288fa"
Oct 30 21:30:03.486581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.486517    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" podStartSLOduration=3.486498258 podStartE2EDuration="3.486498258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:03.482096598 +0000 UTC m=+3111.596830792" watchObservedRunningTime="2026-10-30 21:30:03.486498258 +0000 UTC m=+3111.601232450"
Oct 30 21:30:03.499187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.499123    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" podStartSLOduration=3.499108406 podStartE2EDuration="3.499108406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:03.498538188 +0000 UTC m=+3111.613272377" watchObservedRunningTime="2026-10-30 21:30:03.499108406 +0000 UTC m=+3111.613842597"
Oct 30 21:30:03.540250 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.540005    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" podStartSLOduration=3.539987596 podStartE2EDuration="3.539987596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:03.537614845 +0000 UTC m=+3111.652349034" watchObservedRunningTime="2026-10-30 21:30:03.539987596 +0000 UTC m=+3111.654721787"
Oct 30 21:30:03.561417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.561292    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" podStartSLOduration=3.561269737 podStartE2EDuration="3.561269737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:03.560347147 +0000 UTC m=+3111.675081337" watchObservedRunningTime="2026-10-30 21:30:03.561269737 +0000 UTC m=+3111.676003928"
Oct 30 21:30:03.933714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:03.933666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:04.007680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.007644    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" path="/var/lib/kubelet/pods/1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e/volumes"
Oct 30 21:30:04.008217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.008202    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" path="/var/lib/kubelet/pods/81e9db27-0286-4be5-aa15-22c15b7183cf/volumes"
Oct 30 21:30:04.008694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.008680    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" path="/var/lib/kubelet/pods/90d3a5f7-4ba6-441f-a79c-cc326998db8e/volumes"
Oct 30 21:30:04.458942 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.458910    4852 generic.go:356] "Generic (PLEG): container finished" podID="323e2d2f-adce-4489-81f6-159d46f03848" containerID="9e200f51c1f0e920611a510dca385bd0da7484b2ba3bbba0fd263beb1f30bb46" exitCode=60
Oct 30 21:30:04.459346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.459005    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerDied","Data":"9e200f51c1f0e920611a510dca385bd0da7484b2ba3bbba0fd263beb1f30bb46"}
Oct 30 21:30:04.459346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.459286    4852 scope.go:122] "RemoveContainer" containerID="9e200f51c1f0e920611a510dca385bd0da7484b2ba3bbba0fd263beb1f30bb46"
Oct 30 21:30:04.461316 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.461292    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerStarted","Data":"64b7f31db75245d8d4daef1c290fc25aae1172d72aa722046ea0a929ce1db687"}
Oct 30 21:30:04.463465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.463442    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerStarted","Data":"a39aea8b91782267eeb0e44c9834e093798dd8507ad30b204e2c66a9a53b8bfb"}
Oct 30 21:30:04.465880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.465859    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" event={"ID":"99126667-d257-4026-8b8c-82c227692301","Type":"ContainerStarted","Data":"04c519cdfb4b530918b771599cdf5bd2176972843b89c9a26f53ca5410454b42"}
Oct 30 21:30:04.467597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.467566    4852 generic.go:356] "Generic (PLEG): container finished" podID="2d5c57b1-f837-4b73-8862-b345dd150076" containerID="22f079279723ce8ed20a2dc5aab9b15971aab5b12b29d24fd68bbaa10d1aff5c" exitCode=60
Oct 30 21:30:04.467703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.467646    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" event={"ID":"2d5c57b1-f837-4b73-8862-b345dd150076","Type":"ContainerDied","Data":"22f079279723ce8ed20a2dc5aab9b15971aab5b12b29d24fd68bbaa10d1aff5c"}
Oct 30 21:30:04.467880 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.467859    4852 scope.go:122] "RemoveContainer" containerID="22f079279723ce8ed20a2dc5aab9b15971aab5b12b29d24fd68bbaa10d1aff5c"
Oct 30 21:30:04.469840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.469816    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerStarted","Data":"279574326c81a0d7773ffcf364192e4246cadae3bfcb16bd979e955551b76821"}
Oct 30 21:30:04.471934 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.471912    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerStarted","Data":"4b1575aac4129013238061f94fa8e8713ea57fa3df9c3b8d6c8278f27ab855a5"}
Oct 30 21:30:04.474773 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.474750    4852 generic.go:356] "Generic (PLEG): container finished" podID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerID="3e11f304bb75743e23ba82db8d5f91fe874f6d4fedb158c1b28c816953a30add" exitCode=60
Oct 30 21:30:04.474868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.474833    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerDied","Data":"3e11f304bb75743e23ba82db8d5f91fe874f6d4fedb158c1b28c816953a30add"}
Oct 30 21:30:04.475138 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.475118    4852 scope.go:122] "RemoveContainer" containerID="3e11f304bb75743e23ba82db8d5f91fe874f6d4fedb158c1b28c816953a30add"
Oct 30 21:30:04.478905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.478881    4852 generic.go:356] "Generic (PLEG): container finished" podID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerID="ee647b7aff5e8de3989fbda140400a162208080623814b8a3021ac1c53db9b23" exitCode=60
Oct 30 21:30:04.479030 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.478916    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerDied","Data":"ee647b7aff5e8de3989fbda140400a162208080623814b8a3021ac1c53db9b23"}
Oct 30 21:30:04.479518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.479496    4852 scope.go:122] "RemoveContainer" containerID="ee647b7aff5e8de3989fbda140400a162208080623814b8a3021ac1c53db9b23"
Oct 30 21:30:04.572186 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.572124    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" podStartSLOduration=4.572108675 podStartE2EDuration="4.572108675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:04.571835433 +0000 UTC m=+3112.686569623" watchObservedRunningTime="2026-10-30 21:30:04.572108675 +0000 UTC m=+3112.686842865"
Oct 30 21:30:04.733869 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:04.733832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:05.433032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.432951    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:05.521415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.515706    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" event={"ID":"2d5c57b1-f837-4b73-8862-b345dd150076","Type":"ContainerStarted","Data":"3f62bc27ef6b532ee8509d1fa7a6fa1375c24c1e5c732259cc95b87548b8b744"}
Oct 30 21:30:05.521415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.520218    4852 generic.go:356] "Generic (PLEG): container finished" podID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerID="279574326c81a0d7773ffcf364192e4246cadae3bfcb16bd979e955551b76821" exitCode=60
Oct 30 21:30:05.521415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.520424    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerDied","Data":"279574326c81a0d7773ffcf364192e4246cadae3bfcb16bd979e955551b76821"}
Oct 30 21:30:05.521415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.520504    4852 scope.go:122] "RemoveContainer" containerID="f95c154e637302cfee4a9b27ea95a0596a5ced6ca6d20e4074064b57e49850c0"
Oct 30 21:30:05.521415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.521263    4852 scope.go:122] "RemoveContainer" containerID="279574326c81a0d7773ffcf364192e4246cadae3bfcb16bd979e955551b76821"
Oct 30 21:30:05.522093 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:05.521816    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p_odh-ai-gateway-infra(f921d6df-789f-4dca-8a1e-14c3d443fbc7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7"
Oct 30 21:30:05.526609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.526577    4852 generic.go:356] "Generic (PLEG): container finished" podID="cec79507-2620-4011-9ee9-04547f6810de" containerID="4b1575aac4129013238061f94fa8e8713ea57fa3df9c3b8d6c8278f27ab855a5" exitCode=60
Oct 30 21:30:05.526771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.526679    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerDied","Data":"4b1575aac4129013238061f94fa8e8713ea57fa3df9c3b8d6c8278f27ab855a5"}
Oct 30 21:30:05.527419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.527363    4852 scope.go:122] "RemoveContainer" containerID="4b1575aac4129013238061f94fa8e8713ea57fa3df9c3b8d6c8278f27ab855a5"
Oct 30 21:30:05.527908 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:05.527879    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92_odh-ai-gateway-infra(cec79507-2620-4011-9ee9-04547f6810de)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" podUID="cec79507-2620-4011-9ee9-04547f6810de"
Oct 30 21:30:05.530319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.530283    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerStarted","Data":"5c9d81d89c3e35b030638b6a598b311972775d2995abc8bd07acc51ec4161bb1"}
Oct 30 21:30:05.532704 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.532680    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerStarted","Data":"cc300da8f7c90bccd5ff979af831f59e0b84a99c6ef1d3f2393b59d48f086ded"}
Oct 30 21:30:05.534895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.534869    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerStarted","Data":"a933d5fceec4a613e958a237a2c589f7bee50377012202e4a4cd9249db26720b"}
Oct 30 21:30:05.538164 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.538126    4852 generic.go:356] "Generic (PLEG): container finished" podID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerID="64b7f31db75245d8d4daef1c290fc25aae1172d72aa722046ea0a929ce1db687" exitCode=60
Oct 30 21:30:05.538289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.538181    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerDied","Data":"64b7f31db75245d8d4daef1c290fc25aae1172d72aa722046ea0a929ce1db687"}
Oct 30 21:30:05.538586 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.538565    4852 scope.go:122] "RemoveContainer" containerID="64b7f31db75245d8d4daef1c290fc25aae1172d72aa722046ea0a929ce1db687"
Oct 30 21:30:05.539026 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:05.538952    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt_odh-ai-gateway-infra(0066ff78-850a-4603-8eaa-0b533e03f28d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d"
Oct 30 21:30:05.540105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.540081    4852 scope.go:122] "RemoveContainer" containerID="d17715205f185723b7dad4aa54354719c54a640313af615a1df5e09eeee63137"
Oct 30 21:30:05.540766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.540742    4852 generic.go:356] "Generic (PLEG): container finished" podID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerID="a39aea8b91782267eeb0e44c9834e093798dd8507ad30b204e2c66a9a53b8bfb" exitCode=60
Oct 30 21:30:05.540891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.540871    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerDied","Data":"a39aea8b91782267eeb0e44c9834e093798dd8507ad30b204e2c66a9a53b8bfb"}
Oct 30 21:30:05.541115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.541096    4852 scope.go:122] "RemoveContainer" containerID="a39aea8b91782267eeb0e44c9834e093798dd8507ad30b204e2c66a9a53b8bfb"
Oct 30 21:30:05.541518 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:05.541493    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w_odh-ai-gateway-infra(40028a96-aedb-4c5a-9651-f1784ba3f52f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f"
Oct 30 21:30:05.542969 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.542944    4852 generic.go:356] "Generic (PLEG): container finished" podID="99126667-d257-4026-8b8c-82c227692301" containerID="04c519cdfb4b530918b771599cdf5bd2176972843b89c9a26f53ca5410454b42" exitCode=60
Oct 30 21:30:05.543063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.542996    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" event={"ID":"99126667-d257-4026-8b8c-82c227692301","Type":"ContainerDied","Data":"04c519cdfb4b530918b771599cdf5bd2176972843b89c9a26f53ca5410454b42"}
Oct 30 21:30:05.543229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.543213    4852 scope.go:122] "RemoveContainer" containerID="04c519cdfb4b530918b771599cdf5bd2176972843b89c9a26f53ca5410454b42"
Oct 30 21:30:05.543543 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:05.543524    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj_odh-ai-gateway-infra(99126667-d257-4026-8b8c-82c227692301)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" podUID="99126667-d257-4026-8b8c-82c227692301"
Oct 30 21:30:05.557830 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.557805    4852 scope.go:122] "RemoveContainer" containerID="4fbcc8b2ad419aa45655620627312ddd0317427d3d23509e3fb6a980bc08595b"
Oct 30 21:30:05.579849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.579824    4852 scope.go:122] "RemoveContainer" containerID="f61952e4feaeab82d10494e8424d1ea2b0b3d7e71c0bd72ba4cc88ef3cd288fa"
Oct 30 21:30:05.595474 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.595449    4852 scope.go:122] "RemoveContainer" containerID="73dfdce0eed9f3d5d900ae687a66c2b1ad64e44ce760123e92d27b57300aedfe"
Oct 30 21:30:05.646360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.646328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:05.804243 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.804204    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:05.903678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:05.903644    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:06.354873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.354832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:06.554333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.554306    4852 scope.go:122] "RemoveContainer" containerID="04c519cdfb4b530918b771599cdf5bd2176972843b89c9a26f53ca5410454b42"
Oct 30 21:30:06.554785 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.554756    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj_odh-ai-gateway-infra(99126667-d257-4026-8b8c-82c227692301)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" podUID="99126667-d257-4026-8b8c-82c227692301"
Oct 30 21:30:06.556207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.556179    4852 generic.go:356] "Generic (PLEG): container finished" podID="2d5c57b1-f837-4b73-8862-b345dd150076" containerID="3f62bc27ef6b532ee8509d1fa7a6fa1375c24c1e5c732259cc95b87548b8b744" exitCode=60
Oct 30 21:30:06.556323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.556261    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" event={"ID":"2d5c57b1-f837-4b73-8862-b345dd150076","Type":"ContainerDied","Data":"3f62bc27ef6b532ee8509d1fa7a6fa1375c24c1e5c732259cc95b87548b8b744"}
Oct 30 21:30:06.556427 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.556326    4852 scope.go:122] "RemoveContainer" containerID="22f079279723ce8ed20a2dc5aab9b15971aab5b12b29d24fd68bbaa10d1aff5c"
Oct 30 21:30:06.556626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.556605    4852 scope.go:122] "RemoveContainer" containerID="3f62bc27ef6b532ee8509d1fa7a6fa1375c24c1e5c732259cc95b87548b8b744"
Oct 30 21:30:06.557050 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.557022    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz_odh-ai-gateway-infra(2d5c57b1-f837-4b73-8862-b345dd150076)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" podUID="2d5c57b1-f837-4b73-8862-b345dd150076"
Oct 30 21:30:06.558330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.558313    4852 scope.go:122] "RemoveContainer" containerID="279574326c81a0d7773ffcf364192e4246cadae3bfcb16bd979e955551b76821"
Oct 30 21:30:06.558830 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.558795    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p_odh-ai-gateway-infra(f921d6df-789f-4dca-8a1e-14c3d443fbc7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7"
Oct 30 21:30:06.560219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.560202    4852 scope.go:122] "RemoveContainer" containerID="4b1575aac4129013238061f94fa8e8713ea57fa3df9c3b8d6c8278f27ab855a5"
Oct 30 21:30:06.560566 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.560544    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92_odh-ai-gateway-infra(cec79507-2620-4011-9ee9-04547f6810de)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" podUID="cec79507-2620-4011-9ee9-04547f6810de"
Oct 30 21:30:06.562126 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.562101    4852 generic.go:356] "Generic (PLEG): container finished" podID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerID="5c9d81d89c3e35b030638b6a598b311972775d2995abc8bd07acc51ec4161bb1" exitCode=60
Oct 30 21:30:06.562231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.562138    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerDied","Data":"5c9d81d89c3e35b030638b6a598b311972775d2995abc8bd07acc51ec4161bb1"}
Oct 30 21:30:06.562481 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.562458    4852 scope.go:122] "RemoveContainer" containerID="5c9d81d89c3e35b030638b6a598b311972775d2995abc8bd07acc51ec4161bb1"
Oct 30 21:30:06.562827 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.562799    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g_odh-ai-gateway-infra(eebf6d42-2bac-44f1-bbd5-f7bc24981893)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893"
Oct 30 21:30:06.566158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.566132    4852 generic.go:356] "Generic (PLEG): container finished" podID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerID="cc300da8f7c90bccd5ff979af831f59e0b84a99c6ef1d3f2393b59d48f086ded" exitCode=60
Oct 30 21:30:06.566266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.566206    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerDied","Data":"cc300da8f7c90bccd5ff979af831f59e0b84a99c6ef1d3f2393b59d48f086ded"}
Oct 30 21:30:06.566470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.566452    4852 scope.go:122] "RemoveContainer" containerID="cc300da8f7c90bccd5ff979af831f59e0b84a99c6ef1d3f2393b59d48f086ded"
Oct 30 21:30:06.566756 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.566730    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn_odh-ai-gateway-infra(57bcee80-0a44-4c57-bc04-19616abc74ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff"
Oct 30 21:30:06.568301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.568282    4852 generic.go:356] "Generic (PLEG): container finished" podID="323e2d2f-adce-4489-81f6-159d46f03848" containerID="a933d5fceec4a613e958a237a2c589f7bee50377012202e4a4cd9249db26720b" exitCode=60
Oct 30 21:30:06.568442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.568355    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerDied","Data":"a933d5fceec4a613e958a237a2c589f7bee50377012202e4a4cd9249db26720b"}
Oct 30 21:30:06.568637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.568617    4852 scope.go:122] "RemoveContainer" containerID="a933d5fceec4a613e958a237a2c589f7bee50377012202e4a4cd9249db26720b"
Oct 30 21:30:06.568920 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.568898    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr_odh-ai-gateway-infra(323e2d2f-adce-4489-81f6-159d46f03848)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" podUID="323e2d2f-adce-4489-81f6-159d46f03848"
Oct 30 21:30:06.570539 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.570521    4852 scope.go:122] "RemoveContainer" containerID="64b7f31db75245d8d4daef1c290fc25aae1172d72aa722046ea0a929ce1db687"
Oct 30 21:30:06.570833 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.570805    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt_odh-ai-gateway-infra(0066ff78-850a-4603-8eaa-0b533e03f28d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d"
Oct 30 21:30:06.572655 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.572636    4852 scope.go:122] "RemoveContainer" containerID="a39aea8b91782267eeb0e44c9834e093798dd8507ad30b204e2c66a9a53b8bfb"
Oct 30 21:30:06.572874 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.572855    4852 scope.go:122] "RemoveContainer" containerID="3e11f304bb75743e23ba82db8d5f91fe874f6d4fedb158c1b28c816953a30add"
Oct 30 21:30:06.572942 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:06.572888    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w_odh-ai-gateway-infra(40028a96-aedb-4c5a-9651-f1784ba3f52f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f"
Oct 30 21:30:06.587217 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.587196    4852 scope.go:122] "RemoveContainer" containerID="ee647b7aff5e8de3989fbda140400a162208080623814b8a3021ac1c53db9b23"
Oct 30 21:30:06.602042 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:06.602023    4852 scope.go:122] "RemoveContainer" containerID="9e200f51c1f0e920611a510dca385bd0da7484b2ba3bbba0fd263beb1f30bb46"
Oct 30 21:30:07.251829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:07.251771    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:07.584838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:07.584803    4852 scope.go:122] "RemoveContainer" containerID="3f62bc27ef6b532ee8509d1fa7a6fa1375c24c1e5c732259cc95b87548b8b744"
Oct 30 21:30:07.585308 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:07.585211    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz_odh-ai-gateway-infra(2d5c57b1-f837-4b73-8862-b345dd150076)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" podUID="2d5c57b1-f837-4b73-8862-b345dd150076"
Oct 30 21:30:07.591328 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:07.588157    4852 scope.go:122] "RemoveContainer" containerID="5c9d81d89c3e35b030638b6a598b311972775d2995abc8bd07acc51ec4161bb1"
Oct 30 21:30:07.591328 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:07.588863    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g_odh-ai-gateway-infra(eebf6d42-2bac-44f1-bbd5-f7bc24981893)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893"
Oct 30 21:30:07.594702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:07.594665    4852 scope.go:122] "RemoveContainer" containerID="cc300da8f7c90bccd5ff979af831f59e0b84a99c6ef1d3f2393b59d48f086ded"
Oct 30 21:30:07.595179 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:07.595146    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn_odh-ai-gateway-infra(57bcee80-0a44-4c57-bc04-19616abc74ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff"
Oct 30 21:30:07.596904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:07.596883    4852 scope.go:122] "RemoveContainer" containerID="a933d5fceec4a613e958a237a2c589f7bee50377012202e4a4cd9249db26720b"
Oct 30 21:30:07.597522 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:07.597493    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr_odh-ai-gateway-infra(323e2d2f-adce-4489-81f6-159d46f03848)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" podUID="323e2d2f-adce-4489-81f6-159d46f03848"
Oct 30 21:30:08.094788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.094751    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:08.923502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.923466    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:08.959697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.959655    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"]
Oct 30 21:30:08.964450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.964424    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"]
Oct 30 21:30:08.965213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965197    4852 cpu_manager.go:414] "containerMap: removing container" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy"
Oct 30 21:30:08.965269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965215    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy"
Oct 30 21:30:08.965269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965226    4852 cpu_manager.go:414] "containerMap: removing container" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy"
Oct 30 21:30:08.965269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965232    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy"
Oct 30 21:30:08.965269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965245    4852 cpu_manager.go:414] "containerMap: removing container" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy"
Oct 30 21:30:08.965269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965250    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy"
Oct 30 21:30:08.965459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965358    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="1aee0c1b-24db-4a6e-bbe6-8c3bdb151b4e" containerName="istio-proxy"
Oct 30 21:30:08.965459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965376    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="81e9db27-0286-4be5-aa15-22c15b7183cf" containerName="istio-proxy"
Oct 30 21:30:08.965459 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.965395    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="90d3a5f7-4ba6-441f-a79c-cc326998db8e" containerName="istio-proxy"
Oct 30 21:30:08.970831 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:08.970810    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.015679 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.015642    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:30:09.030782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.025380    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"]
Oct 30 21:30:09.040112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.039666    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.040112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.039822    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.040112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.039945    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzsd9\" (UniqueName: \"kubernetes.io/projected/e5bd3453-1c90-4cb4-b648-5484b42fa217-kube-api-access-zzsd9\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.136004 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.135972    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:09.141447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.141420    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.141583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.141487    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.141639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.141579    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzsd9\" (UniqueName: \"kubernetes.io/projected/e5bd3453-1c90-4cb4-b648-5484b42fa217-kube-api-access-zzsd9\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.142177 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.142157    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.143878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.143854    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.155712 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.155681    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzsd9\" (UniqueName: \"kubernetes.io/projected/e5bd3453-1c90-4cb4-b648-5484b42fa217-kube-api-access-zzsd9\") pod \"maas-api-revoke-sub-b8673e82-cg8td\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.256485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.227287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:09.286693 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.286639    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:09.391799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.382232    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d5c57b1-f837-4b73-8862-b345dd150076-kube-api-access-7l87m\" (UniqueName: \"kubernetes.io/projected/2d5c57b1-f837-4b73-8862-b345dd150076-kube-api-access-7l87m\") pod \"2d5c57b1-f837-4b73-8862-b345dd150076\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") "
Oct 30 21:30:09.391799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.382847    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d5c57b1-f837-4b73-8862-b345dd150076-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c57b1-f837-4b73-8862-b345dd150076-maas-api-service-ca\") pod \"2d5c57b1-f837-4b73-8862-b345dd150076\" (UID: \"2d5c57b1-f837-4b73-8862-b345dd150076\") "
Oct 30 21:30:09.392907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.392707    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:09.425353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.414356    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d5c57b1-f837-4b73-8862-b345dd150076-maas-api-service-ca" pod "2d5c57b1-f837-4b73-8862-b345dd150076" (UID: "2d5c57b1-f837-4b73-8862-b345dd150076"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:09.451527 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.451454    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5c57b1-f837-4b73-8862-b345dd150076-kube-api-access-7l87m" pod "2d5c57b1-f837-4b73-8862-b345dd150076" (UID: "2d5c57b1-f837-4b73-8862-b345dd150076"). InnerVolumeSpecName "kube-api-access-7l87m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:09.472455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.471504    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"]
Oct 30 21:30:09.472455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.471943    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy" containerID="cri-o://a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e" gracePeriod=30
Oct 30 21:30:09.488610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.488577    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l87m\" (UniqueName: \"kubernetes.io/projected/2d5c57b1-f837-4b73-8862-b345dd150076-kube-api-access-7l87m\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:09.488610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.488606    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c57b1-f837-4b73-8862-b345dd150076-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:09.514945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.514860    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:09.624424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.622781    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz" event={"ID":"2d5c57b1-f837-4b73-8862-b345dd150076","Type":"ContainerDied","Data":"386b55e2afef16b6f4c82bcc57c5f3153f8384fdb4c7e239f2b51916cbb0621b"}
Oct 30 21:30:09.624424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.622842    4852 scope.go:122] "RemoveContainer" containerID="3f62bc27ef6b532ee8509d1fa7a6fa1375c24c1e5c732259cc95b87548b8b744"
Oct 30 21:30:09.624424 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.622947    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"
Oct 30 21:30:09.662549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.662514    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"]
Oct 30 21:30:09.667560 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.667525    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ffce13-29889930-74xwz"]
Oct 30 21:30:09.775811 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:09.775778    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5bd3453_1c90_4cb4_b648_5484b42fa217.slice/crio-bb068c81dace3002100f5130a0f712b114a32fd29f256261d1c9f9c86774d8df WatchSource:0}: Error finding container bb068c81dace3002100f5130a0f712b114a32fd29f256261d1c9f9c86774d8df: Status 404 returned error can't find the container with id bb068c81dace3002100f5130a0f712b114a32fd29f256261d1c9f9c86774d8df
Oct 30 21:30:09.775944 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.775817    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"]
Oct 30 21:30:09.958632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:09.958600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:10.006650 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:10.006616    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d5c57b1-f837-4b73-8862-b345dd150076" path="/var/lib/kubelet/pods/2d5c57b1-f837-4b73-8862-b345dd150076/volumes"
Oct 30 21:30:10.634654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:10.634610    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td" event={"ID":"e5bd3453-1c90-4cb4-b648-5484b42fa217","Type":"ContainerStarted","Data":"2ef1a32302c5dc9bed9885e873ac0f56085a170529f15d31ebaa4eefe8773619"}
Oct 30 21:30:10.634871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:10.634686    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td" event={"ID":"e5bd3453-1c90-4cb4-b648-5484b42fa217","Type":"ContainerStarted","Data":"bb068c81dace3002100f5130a0f712b114a32fd29f256261d1c9f9c86774d8df"}
Oct 30 21:30:10.661404 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:10.661340    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td" podStartSLOduration=2.661323773 podStartE2EDuration="2.661323773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:10.660523331 +0000 UTC m=+3118.775257520" watchObservedRunningTime="2026-10-30 21:30:10.661323773 +0000 UTC m=+3118.776057964"
Oct 30 21:30:10.675364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:10.675334    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:11.278446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.278413    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"]
Oct 30 21:30:11.278882 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.278641    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t" podUID="e2d9cd02-b0eb-4e2a-a614-353161d08efc" containerName="maas-api" containerID="cri-o://cabab92c5416b0774fcd203437f722acf371463e75e5acb284c64f3454d7ace3" gracePeriod=30
Oct 30 21:30:11.293239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.293209    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:11.328646 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.328605    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"]
Oct 30 21:30:11.511566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.511454    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:11.624617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.622495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:11.645443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.645032    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:11.769302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.760640    4852 generic.go:356] "Generic (PLEG): container finished" podID="e2d9cd02-b0eb-4e2a-a614-353161d08efc" containerID="cabab92c5416b0774fcd203437f722acf371463e75e5acb284c64f3454d7ace3" exitCode=0
Oct 30 21:30:11.769302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.760774    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t" event={"ID":"e2d9cd02-b0eb-4e2a-a614-353161d08efc","Type":"ContainerDied","Data":"cabab92c5416b0774fcd203437f722acf371463e75e5acb284c64f3454d7ace3"}
Oct 30 21:30:11.804889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.804105    4852 generic.go:356] "Generic (PLEG): container finished" podID="e5bd3453-1c90-4cb4-b648-5484b42fa217" containerID="2ef1a32302c5dc9bed9885e873ac0f56085a170529f15d31ebaa4eefe8773619" exitCode=0
Oct 30 21:30:11.804889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.804243    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td" event={"ID":"e5bd3453-1c90-4cb4-b648-5484b42fa217","Type":"ContainerDied","Data":"2ef1a32302c5dc9bed9885e873ac0f56085a170529f15d31ebaa4eefe8773619"}
Oct 30 21:30:11.838470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.834921    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"
Oct 30 21:30:11.838470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.834920    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj" event={"ID":"99126667-d257-4026-8b8c-82c227692301","Type":"ContainerDied","Data":"3a1ca8be5690cf77f35248c11c56f68c59ea1c0bce6c1d5aaf074450a06d6db8"}
Oct 30 21:30:11.838470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.835145    4852 scope.go:122] "RemoveContainer" containerID="04c519cdfb4b530918b771599cdf5bd2176972843b89c9a26f53ca5410454b42"
Oct 30 21:30:11.864035 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.863994    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99126667-d257-4026-8b8c-82c227692301-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99126667-d257-4026-8b8c-82c227692301-maas-api-service-ca\") pod \"99126667-d257-4026-8b8c-82c227692301\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") "
Oct 30 21:30:11.864260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.864183    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99126667-d257-4026-8b8c-82c227692301-kube-api-access-wwvfc\" (UniqueName: \"kubernetes.io/projected/99126667-d257-4026-8b8c-82c227692301-kube-api-access-wwvfc\") pod \"99126667-d257-4026-8b8c-82c227692301\" (UID: \"99126667-d257-4026-8b8c-82c227692301\") "
Oct 30 21:30:11.864787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.864762    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99126667-d257-4026-8b8c-82c227692301-maas-api-service-ca" pod "99126667-d257-4026-8b8c-82c227692301" (UID: "99126667-d257-4026-8b8c-82c227692301"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:11.864914 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.864888    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99126667-d257-4026-8b8c-82c227692301-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:11.867037 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.867011    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99126667-d257-4026-8b8c-82c227692301-kube-api-access-wwvfc" pod "99126667-d257-4026-8b8c-82c227692301" (UID: "99126667-d257-4026-8b8c-82c227692301"). InnerVolumeSpecName "kube-api-access-wwvfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:11.882894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.882864    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:30:11.966214 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.966175    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-maas-api-tls\") pod \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") "
Oct 30 21:30:11.966462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.966263    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2d9cd02-b0eb-4e2a-a614-353161d08efc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d9cd02-b0eb-4e2a-a614-353161d08efc-openshift-service-ca\") pod \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") "
Oct 30 21:30:11.966462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.966355    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2d9cd02-b0eb-4e2a-a614-353161d08efc-kube-api-access-2j285\" (UniqueName: \"kubernetes.io/projected/e2d9cd02-b0eb-4e2a-a614-353161d08efc-kube-api-access-2j285\") pod \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") "
Oct 30 21:30:11.966856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.966821    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d9cd02-b0eb-4e2a-a614-353161d08efc-openshift-service-ca" pod "e2d9cd02-b0eb-4e2a-a614-353161d08efc" (UID: "e2d9cd02-b0eb-4e2a-a614-353161d08efc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:11.967366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.967336    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwvfc\" (UniqueName: \"kubernetes.io/projected/99126667-d257-4026-8b8c-82c227692301-kube-api-access-wwvfc\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:11.967508 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.967371    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2d9cd02-b0eb-4e2a-a614-353161d08efc-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:11.968590 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.968560    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-maas-api-tls" pod "e2d9cd02-b0eb-4e2a-a614-353161d08efc" (UID: "e2d9cd02-b0eb-4e2a-a614-353161d08efc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:11.969142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:11.969112    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d9cd02-b0eb-4e2a-a614-353161d08efc-kube-api-access-2j285" pod "e2d9cd02-b0eb-4e2a-a614-353161d08efc" (UID: "e2d9cd02-b0eb-4e2a-a614-353161d08efc"). InnerVolumeSpecName "kube-api-access-2j285". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:12.046507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.046468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:12.068518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.068427    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-metrics-certs\") pod \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\" (UID: \"e2d9cd02-b0eb-4e2a-a614-353161d08efc\") "
Oct 30 21:30:12.069008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.068987    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j285\" (UniqueName: \"kubernetes.io/projected/e2d9cd02-b0eb-4e2a-a614-353161d08efc-kube-api-access-2j285\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:12.069103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.069012    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:12.071034 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.071011    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-metrics-certs" pod "e2d9cd02-b0eb-4e2a-a614-353161d08efc" (UID: "e2d9cd02-b0eb-4e2a-a614-353161d08efc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:12.154580 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.154544    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"]
Oct 30 21:30:12.159323 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.159290    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-2ac526-29889930-ssmtj"]
Oct 30 21:30:12.170005 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.169974    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d9cd02-b0eb-4e2a-a614-353161d08efc-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:12.637534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.637502    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:12.848624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.848593    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"
Oct 30 21:30:12.848825 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.848625    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t" event={"ID":"e2d9cd02-b0eb-4e2a-a614-353161d08efc","Type":"ContainerDied","Data":"e2abe7ab870b744e2c08cf54e380e677cd1a0e8be96abe2c3952803dd4504297"}
Oct 30 21:30:12.848825 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.848702    4852 scope.go:122] "RemoveContainer" containerID="cabab92c5416b0774fcd203437f722acf371463e75e5acb284c64f3454d7ace3"
Oct 30 21:30:12.907758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.907687    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"]
Oct 30 21:30:12.911906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:12.911866    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2ac526-7647f45775-sr85t"]
Oct 30 21:30:13.065900 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.065872    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:13.089008 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.088974    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-openshift-ca\") pod \"e5bd3453-1c90-4cb4-b648-5484b42fa217\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") "
Oct 30 21:30:13.089252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.089228    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-xks-ca\") pod \"e5bd3453-1c90-4cb4-b648-5484b42fa217\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") "
Oct 30 21:30:13.089510 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.089485    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5bd3453-1c90-4cb4-b648-5484b42fa217-kube-api-access-zzsd9\" (UniqueName: \"kubernetes.io/projected/e5bd3453-1c90-4cb4-b648-5484b42fa217-kube-api-access-zzsd9\") pod \"e5bd3453-1c90-4cb4-b648-5484b42fa217\" (UID: \"e5bd3453-1c90-4cb4-b648-5484b42fa217\") "
Oct 30 21:30:13.089950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.089366    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-openshift-ca" pod "e5bd3453-1c90-4cb4-b648-5484b42fa217" (UID: "e5bd3453-1c90-4cb4-b648-5484b42fa217"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:13.090548 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.090521    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:13.096632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.096603    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5bd3453-1c90-4cb4-b648-5484b42fa217-kube-api-access-zzsd9" pod "e5bd3453-1c90-4cb4-b648-5484b42fa217" (UID: "e5bd3453-1c90-4cb4-b648-5484b42fa217"). InnerVolumeSpecName "kube-api-access-zzsd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:13.102566 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.102539    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-xks-ca" pod "e5bd3453-1c90-4cb4-b648-5484b42fa217" (UID: "e5bd3453-1c90-4cb4-b648-5484b42fa217"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:13.192441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.192328    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e5bd3453-1c90-4cb4-b648-5484b42fa217-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:13.192441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.192364    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzsd9\" (UniqueName: \"kubernetes.io/projected/e5bd3453-1c90-4cb4-b648-5484b42fa217-kube-api-access-zzsd9\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:13.422777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.422737    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:13.504702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.501379    4852 patch_prober.go:29] interesting pod/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:30:13.504702 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.501467    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:30:13.861722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.861691    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"
Oct 30 21:30:13.862106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.861725    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td" event={"ID":"e5bd3453-1c90-4cb4-b648-5484b42fa217","Type":"ContainerDied","Data":"bb068c81dace3002100f5130a0f712b114a32fd29f256261d1c9f9c86774d8df"}
Oct 30 21:30:13.862106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:13.861798    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb068c81dace3002100f5130a0f712b114a32fd29f256261d1c9f9c86774d8df"
Oct 30 21:30:14.006366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.006332    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99126667-d257-4026-8b8c-82c227692301" path="/var/lib/kubelet/pods/99126667-d257-4026-8b8c-82c227692301/volumes"
Oct 30 21:30:14.006865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.006846    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2d9cd02-b0eb-4e2a-a614-353161d08efc" path="/var/lib/kubelet/pods/e2d9cd02-b0eb-4e2a-a614-353161d08efc/volumes"
Oct 30 21:30:14.145440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.145305    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:14.776839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.776808    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823334    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-certs\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823565    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-istio-token\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823605    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-socket\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823640    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-kube-api-access-dfn8x\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-kube-api-access-dfn8x\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823752    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-credential-socket\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823787    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-envoy\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823895    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-data\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823940    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c4f7322-e13b-41bc-b47b-39390953298e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f7322-e13b-41bc-b47b-39390953298e-istiod-ca-cert\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.823993    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c4f7322-e13b-41bc-b47b-39390953298e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f7322-e13b-41bc-b47b-39390953298e-istio-podinfo\") pod \"3c4f7322-e13b-41bc-b47b-39390953298e\" (UID: \"3c4f7322-e13b-41bc-b47b-39390953298e\") "
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.824229    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-credential-socket" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.824498    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-socket" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.824756    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.824779    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.826481    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-data" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.827833    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c4f7322-e13b-41bc-b47b-39390953298e-istiod-ca-cert" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:14.829601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.827895    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-certs" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:14.830874 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.830477    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-istio-token" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:14.833118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.833088    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c4f7322-e13b-41bc-b47b-39390953298e-istio-podinfo" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:14.833243 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.833113    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-kube-api-access-dfn8x" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "kube-api-access-dfn8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:14.833243 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.833211    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-envoy" pod "3c4f7322-e13b-41bc-b47b-39390953298e" (UID: "3c4f7322-e13b-41bc-b47b-39390953298e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:14.852653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.852620    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:14.875056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.875020    4852 generic.go:356] "Generic (PLEG): container finished" podID="3c4f7322-e13b-41bc-b47b-39390953298e" containerID="a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e" exitCode=0
Oct 30 21:30:14.875451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.875088    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" event={"ID":"3c4f7322-e13b-41bc-b47b-39390953298e","Type":"ContainerDied","Data":"a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e"}
Oct 30 21:30:14.875451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.875140    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv" event={"ID":"3c4f7322-e13b-41bc-b47b-39390953298e","Type":"ContainerDied","Data":"85b4c0d84227da70d97bd60b2ec2564bb2b181d732fb8adb0a0ed1f73cfcae0a"}
Oct 30 21:30:14.875451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.875178    4852 scope.go:122] "RemoveContainer" containerID="a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e"
Oct 30 21:30:14.875451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.875305    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"
Oct 30 21:30:14.887690 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.887657    4852 scope.go:122] "RemoveContainer" containerID="55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52"
Oct 30 21:30:14.898650 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.898624    4852 scope.go:122] "RemoveContainer" containerID="a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e"
Oct 30 21:30:14.898963 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:14.898941    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e\": container with ID starting with a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e not found: ID does not exist" containerID="a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e"
Oct 30 21:30:14.899080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.898977    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e"} err="failed to get container status \"a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e\": rpc error: code = NotFound desc = could not find container \"a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e\": container with ID starting with a92ca985cdc04c86a3df7315e107d68a813fe2106ef8c512fd8269f9efc8423e not found: ID does not exist"
Oct 30 21:30:14.899080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.899001    4852 scope.go:122] "RemoveContainer" containerID="55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52"
Oct 30 21:30:14.899256 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:14.899239    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52\": container with ID starting with 55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52 not found: ID does not exist" containerID="55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52"
Oct 30 21:30:14.899309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.899263    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52"} err="failed to get container status \"55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52\": rpc error: code = NotFound desc = could not find container \"55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52\": container with ID starting with 55125fcf00a43b69c1de9147475a70fb60b4c782698cc9b7638b7753fe146e52 not found: ID does not exist"
Oct 30 21:30:14.902683 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.902655    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"]
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.931220    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-e53c37-openshift-default-84d754496f-ktlqv"]
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.934792    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.934852    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfn8x\" (UniqueName: \"kubernetes.io/projected/3c4f7322-e13b-41bc-b47b-39390953298e-kube-api-access-dfn8x\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.934872    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.934886    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.935003    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4f7322-e13b-41bc-b47b-39390953298e-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.935016    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4f7322-e13b-41bc-b47b-39390953298e-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:14.935432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:14.935030    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4f7322-e13b-41bc-b47b-39390953298e-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:15.605664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:15.605631    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:15.998560 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:15.998467    4852 scope.go:122] "RemoveContainer" containerID="a933d5fceec4a613e958a237a2c589f7bee50377012202e4a4cd9249db26720b"
Oct 30 21:30:16.007726 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.007702    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" path="/var/lib/kubelet/pods/3c4f7322-e13b-41bc-b47b-39390953298e/volumes"
Oct 30 21:30:16.412476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.412441    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:16.520515 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.520439    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:16.613533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.613493    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:16.877822 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.877730    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"]
Oct 30 21:30:16.878072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.878020    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy" containerID="cri-o://307698abde47f9535848da222d9c06005cbdb24f1c51ca3d6650c70f2f63b48e" gracePeriod=30
Oct 30 21:30:16.895852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.895815    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:16.897406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:16.897369    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerStarted","Data":"291b14682a9c440a05f19877fdde70ebd5e0dd7d7348630215559aaed6de3bf0"}
Oct 30 21:30:17.001412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.000455    4852 scope.go:122] "RemoveContainer" containerID="a39aea8b91782267eeb0e44c9834e093798dd8507ad30b204e2c66a9a53b8bfb"
Oct 30 21:30:17.021450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.016695    4852 scope.go:122] "RemoveContainer" containerID="279574326c81a0d7773ffcf364192e4246cadae3bfcb16bd979e955551b76821"
Oct 30 21:30:17.558779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.558743    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"]
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559598    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e5bd3453-1c90-4cb4-b648-5484b42fa217" containerName="revoke-keys"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559613    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5bd3453-1c90-4cb4-b648-5484b42fa217" containerName="revoke-keys"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559624    4852 cpu_manager.go:414] "containerMap: removing container" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559629    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559639    4852 cpu_manager.go:414] "containerMap: removing container" podUID="99126667-d257-4026-8b8c-82c227692301" containerName="cleanup"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559645    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99126667-d257-4026-8b8c-82c227692301" containerName="cleanup"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559661    4852 cpu_manager.go:414] "containerMap: removing container" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559666    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559675    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2d5c57b1-f837-4b73-8862-b345dd150076" containerName="cleanup"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559679    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d5c57b1-f837-4b73-8862-b345dd150076" containerName="cleanup"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559687    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2d5c57b1-f837-4b73-8862-b345dd150076" containerName="cleanup"
Oct 30 21:30:17.559706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559691    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d5c57b1-f837-4b73-8862-b345dd150076" containerName="cleanup"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559734    4852 cpu_manager.go:414] "containerMap: removing container" podUID="99126667-d257-4026-8b8c-82c227692301" containerName="cleanup"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559743    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99126667-d257-4026-8b8c-82c227692301" containerName="cleanup"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559759    4852 cpu_manager.go:414] "containerMap: removing container" podUID="e2d9cd02-b0eb-4e2a-a614-353161d08efc" containerName="maas-api"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559763    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2d9cd02-b0eb-4e2a-a614-353161d08efc" containerName="maas-api"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559916    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="99126667-d257-4026-8b8c-82c227692301" containerName="cleanup"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559933    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2d9cd02-b0eb-4e2a-a614-353161d08efc" containerName="maas-api"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559944    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5bd3453-1c90-4cb4-b648-5484b42fa217" containerName="revoke-keys"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559953    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559961    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="99126667-d257-4026-8b8c-82c227692301" containerName="cleanup"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559972    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c4f7322-e13b-41bc-b47b-39390953298e" containerName="istio-proxy"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559980    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d5c57b1-f837-4b73-8862-b345dd150076" containerName="cleanup"
Oct 30 21:30:17.560111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.559987    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d5c57b1-f837-4b73-8862-b345dd150076" containerName="cleanup"
Oct 30 21:30:17.564306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.564288    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.570905 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.570883    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:30:17.572257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.572234    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"]
Oct 30 21:30:17.627795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.627759    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.627953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.627932    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jgxw\" (UniqueName: \"kubernetes.io/projected/c27704f4-9744-46b0-9f3f-eb655a517c21-kube-api-access-5jgxw\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.628017 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.627972    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.712554 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.712464    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:17.728979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.728946    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.729187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.729045    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jgxw\" (UniqueName: \"kubernetes.io/projected/c27704f4-9744-46b0-9f3f-eb655a517c21-kube-api-access-5jgxw\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.729187 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.729076    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.729714 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.729694    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.731483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.731458    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.736940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.736894    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jgxw\" (UniqueName: \"kubernetes.io/projected/c27704f4-9744-46b0-9f3f-eb655a517c21-kube-api-access-5jgxw\") pod \"maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.883016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.882975    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:17.909935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.909900    4852 generic.go:356] "Generic (PLEG): container finished" podID="323e2d2f-adce-4489-81f6-159d46f03848" containerID="291b14682a9c440a05f19877fdde70ebd5e0dd7d7348630215559aaed6de3bf0" exitCode=60
Oct 30 21:30:17.910082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.909965    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerDied","Data":"291b14682a9c440a05f19877fdde70ebd5e0dd7d7348630215559aaed6de3bf0"}
Oct 30 21:30:17.910082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.910022    4852 scope.go:122] "RemoveContainer" containerID="a933d5fceec4a613e958a237a2c589f7bee50377012202e4a4cd9249db26720b"
Oct 30 21:30:17.910611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.910587    4852 scope.go:122] "RemoveContainer" containerID="291b14682a9c440a05f19877fdde70ebd5e0dd7d7348630215559aaed6de3bf0"
Oct 30 21:30:17.911065 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:17.911029    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr_odh-ai-gateway-infra(323e2d2f-adce-4489-81f6-159d46f03848)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" podUID="323e2d2f-adce-4489-81f6-159d46f03848"
Oct 30 21:30:17.913060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.913033    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerStarted","Data":"fde64e3a20d257c40ddef95b93a6255bd89726cd8b7f844e34df6d80e16529a4"}
Oct 30 21:30:17.915647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.915627    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerStarted","Data":"92e2273d1441cad57aa50e00380d32b6e46e258abc830dea8859dc1e342b7171"}
Oct 30 21:30:17.954870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.954835    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"]
Oct 30 21:30:18.000506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.998013    4852 scope.go:122] "RemoveContainer" containerID="cc300da8f7c90bccd5ff979af831f59e0b84a99c6ef1d3f2393b59d48f086ded"
Oct 30 21:30:18.000506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:17.998628    4852 scope.go:122] "RemoveContainer" containerID="64b7f31db75245d8d4daef1c290fc25aae1172d72aa722046ea0a929ce1db687"
Oct 30 21:30:18.177339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.177311    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"]
Oct 30 21:30:18.253806 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:18.253772    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27704f4_9744_46b0_9f3f_eb655a517c21.slice/crio-d04c7e4157c5b50115497ff72139a9dd98efd2d05fff71539a8b80197e074a27 WatchSource:0}: Error finding container d04c7e4157c5b50115497ff72139a9dd98efd2d05fff71539a8b80197e074a27: Status 404 returned error can't find the container with id d04c7e4157c5b50115497ff72139a9dd98efd2d05fff71539a8b80197e074a27
Oct 30 21:30:18.614669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.614635    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:18.925984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.925892    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk" event={"ID":"c27704f4-9744-46b0-9f3f-eb655a517c21","Type":"ContainerStarted","Data":"144ce53a87de912933407e033d7e26a29a0f99c4ccea27ae6a0509af44559e4c"}
Oct 30 21:30:18.926252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.926212    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk" event={"ID":"c27704f4-9744-46b0-9f3f-eb655a517c21","Type":"ContainerStarted","Data":"d04c7e4157c5b50115497ff72139a9dd98efd2d05fff71539a8b80197e074a27"}
Oct 30 21:30:18.928302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.928277    4852 generic.go:356] "Generic (PLEG): container finished" podID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerID="92e2273d1441cad57aa50e00380d32b6e46e258abc830dea8859dc1e342b7171" exitCode=60
Oct 30 21:30:18.928499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.928320    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerDied","Data":"92e2273d1441cad57aa50e00380d32b6e46e258abc830dea8859dc1e342b7171"}
Oct 30 21:30:18.928499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.928381    4852 scope.go:122] "RemoveContainer" containerID="279574326c81a0d7773ffcf364192e4246cadae3bfcb16bd979e955551b76821"
Oct 30 21:30:18.928809 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.928792    4852 scope.go:122] "RemoveContainer" containerID="92e2273d1441cad57aa50e00380d32b6e46e258abc830dea8859dc1e342b7171"
Oct 30 21:30:18.929196 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:18.929173    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p_odh-ai-gateway-infra(f921d6df-789f-4dca-8a1e-14c3d443fbc7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7"
Oct 30 21:30:18.930748 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.930727    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerStarted","Data":"d7981bb461f07a95d1e86af3512242557b3687f03506e96d7eae0c347712cc4d"}
Oct 30 21:30:18.936835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.936799    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerStarted","Data":"94bcf5e30a425fa874e205664435890093accbfe5ca124e19ad683b1c903771f"}
Oct 30 21:30:18.939305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.939277    4852 generic.go:356] "Generic (PLEG): container finished" podID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerID="fde64e3a20d257c40ddef95b93a6255bd89726cd8b7f844e34df6d80e16529a4" exitCode=60
Oct 30 21:30:18.939441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.939315    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerDied","Data":"fde64e3a20d257c40ddef95b93a6255bd89726cd8b7f844e34df6d80e16529a4"}
Oct 30 21:30:18.939860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.939840    4852 scope.go:122] "RemoveContainer" containerID="fde64e3a20d257c40ddef95b93a6255bd89726cd8b7f844e34df6d80e16529a4"
Oct 30 21:30:18.940198 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:18.940180    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w_odh-ai-gateway-infra(40028a96-aedb-4c5a-9651-f1784ba3f52f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f"
Oct 30 21:30:18.946178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.946150    4852 scope.go:122] "RemoveContainer" containerID="a39aea8b91782267eeb0e44c9834e093798dd8507ad30b204e2c66a9a53b8bfb"
Oct 30 21:30:18.949547 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.947432    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk" podStartSLOduration=1.947381434 podStartE2EDuration="1.947381434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:18.944082886 +0000 UTC m=+3127.058817077" watchObservedRunningTime="2026-10-30 21:30:18.947381434 +0000 UTC m=+3127.062115625"
Oct 30 21:30:18.959017 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.958980    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"]
Oct 30 21:30:18.997513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:18.997478    4852 scope.go:122] "RemoveContainer" containerID="4b1575aac4129013238061f94fa8e8713ea57fa3df9c3b8d6c8278f27ab855a5"
Oct 30 21:30:19.000439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.000364    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"]
Oct 30 21:30:19.135251 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.135225    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:19.150266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.150231    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/323e2d2f-adce-4489-81f6-159d46f03848-kube-api-access-gkg74\" (UniqueName: \"kubernetes.io/projected/323e2d2f-adce-4489-81f6-159d46f03848-kube-api-access-gkg74\") pod \"323e2d2f-adce-4489-81f6-159d46f03848\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") "
Oct 30 21:30:19.150482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.150448    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/323e2d2f-adce-4489-81f6-159d46f03848-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/323e2d2f-adce-4489-81f6-159d46f03848-maas-api-service-ca\") pod \"323e2d2f-adce-4489-81f6-159d46f03848\" (UID: \"323e2d2f-adce-4489-81f6-159d46f03848\") "
Oct 30 21:30:19.153647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.153612    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/323e2d2f-adce-4489-81f6-159d46f03848-maas-api-service-ca" pod "323e2d2f-adce-4489-81f6-159d46f03848" (UID: "323e2d2f-adce-4489-81f6-159d46f03848"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:19.251330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.251241    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/323e2d2f-adce-4489-81f6-159d46f03848-kube-api-access-gkg74" pod "323e2d2f-adce-4489-81f6-159d46f03848" (UID: "323e2d2f-adce-4489-81f6-159d46f03848"). InnerVolumeSpecName "kube-api-access-gkg74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:19.254212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.254179    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/323e2d2f-adce-4489-81f6-159d46f03848-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:19.254212 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.254214    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkg74\" (UniqueName: \"kubernetes.io/projected/323e2d2f-adce-4489-81f6-159d46f03848-kube-api-access-gkg74\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:19.265816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.265783    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:19.347317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.347279    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:19.605532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.605499    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:19.716273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.716240    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:19.981468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.978559    4852 generic.go:356] "Generic (PLEG): container finished" podID="c27704f4-9744-46b0-9f3f-eb655a517c21" containerID="144ce53a87de912933407e033d7e26a29a0f99c4ccea27ae6a0509af44559e4c" exitCode=0
Oct 30 21:30:19.981468 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.978644    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk" event={"ID":"c27704f4-9744-46b0-9f3f-eb655a517c21","Type":"ContainerDied","Data":"144ce53a87de912933407e033d7e26a29a0f99c4ccea27ae6a0509af44559e4c"}
Oct 30 21:30:19.990447 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.987690    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerStarted","Data":"ed98f72f408834385ae984f69e5707a993fa9043d5da87b064504122d03afc66"}
Oct 30 21:30:19.993211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.993182    4852 generic.go:356] "Generic (PLEG): container finished" podID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerID="d7981bb461f07a95d1e86af3512242557b3687f03506e96d7eae0c347712cc4d" exitCode=60
Oct 30 21:30:19.993351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.993279    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerDied","Data":"d7981bb461f07a95d1e86af3512242557b3687f03506e96d7eae0c347712cc4d"}
Oct 30 21:30:19.993351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.993334    4852 scope.go:122] "RemoveContainer" containerID="cc300da8f7c90bccd5ff979af831f59e0b84a99c6ef1d3f2393b59d48f086ded"
Oct 30 21:30:19.993829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:19.993811    4852 scope.go:122] "RemoveContainer" containerID="d7981bb461f07a95d1e86af3512242557b3687f03506e96d7eae0c347712cc4d"
Oct 30 21:30:19.994204 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:19.994182    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn_odh-ai-gateway-infra(57bcee80-0a44-4c57-bc04-19616abc74ff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff"
Oct 30 21:30:20.002482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.002146    4852 scope.go:122] "RemoveContainer" containerID="5c9d81d89c3e35b030638b6a598b311972775d2995abc8bd07acc51ec4161bb1"
Oct 30 21:30:20.011975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.011785    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"
Oct 30 21:30:20.022581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.022537    4852 generic.go:356] "Generic (PLEG): container finished" podID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerID="94bcf5e30a425fa874e205664435890093accbfe5ca124e19ad683b1c903771f" exitCode=60
Oct 30 21:30:20.028760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.025984    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr" event={"ID":"323e2d2f-adce-4489-81f6-159d46f03848","Type":"ContainerDied","Data":"015828d30c588f8bcea66753cd94f822d03b7a85ccb7b657f62b5e62c0c2b504"}
Oct 30 21:30:20.028760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.026050    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"]
Oct 30 21:30:20.028760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.026090    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerDied","Data":"94bcf5e30a425fa874e205664435890093accbfe5ca124e19ad683b1c903771f"}
Oct 30 21:30:20.039711 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.039062    4852 scope.go:122] "RemoveContainer" containerID="291b14682a9c440a05f19877fdde70ebd5e0dd7d7348630215559aaed6de3bf0"
Oct 30 21:30:20.063245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.063202    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"]
Oct 30 21:30:20.073616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.073575    4852 scope.go:122] "RemoveContainer" containerID="64b7f31db75245d8d4daef1c290fc25aae1172d72aa722046ea0a929ce1db687"
Oct 30 21:30:20.179155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.178101    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"]
Oct 30 21:30:20.179350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179173    4852 cpu_manager.go:414] "containerMap: removing container" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.179350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179191    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.179350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179204    4852 cpu_manager.go:414] "containerMap: removing container" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.179350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179210    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.179350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179235    4852 cpu_manager.go:414] "containerMap: removing container" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.179350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179240    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.179722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179367    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.179722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.179397    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.184474 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.184451    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"]
Oct 30 21:30:20.184619 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.184560    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.188682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.188365    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"]
Oct 30 21:30:20.191205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.191163    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.191511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.191476    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znr4w\" (UniqueName: \"kubernetes.io/projected/72e54db2-6a16-44e6-a3a0-6b388c371acd-kube-api-access-znr4w\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.191623 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.191586    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.192786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.192766    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-e07442-29889930-xf7nr"]
Oct 30 21:30:20.234722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.234662    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"]
Oct 30 21:30:20.236064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.236037    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="323e2d2f-adce-4489-81f6-159d46f03848" containerName="cleanup"
Oct 30 21:30:20.241000 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.240938    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.268600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.268570    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"]
Oct 30 21:30:20.289841 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.289807    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:20.293314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.293205    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.293314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.293289    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.295519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.295487    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.295651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.295584    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znr4w\" (UniqueName: \"kubernetes.io/projected/72e54db2-6a16-44e6-a3a0-6b388c371acd-kube-api-access-znr4w\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.295708 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.295669    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.295755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.295713    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm4pn\" (UniqueName: \"kubernetes.io/projected/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-kube-api-access-cm4pn\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.297105 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.297077    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.299431 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.299366    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.335266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.329063    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znr4w\" (UniqueName: \"kubernetes.io/projected/72e54db2-6a16-44e6-a3a0-6b388c371acd-kube-api-access-znr4w\") pod \"maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.397300    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f921d6df-789f-4dca-8a1e-14c3d443fbc7-kube-api-access-2vrzr\" (UniqueName: \"kubernetes.io/projected/f921d6df-789f-4dca-8a1e-14c3d443fbc7-kube-api-access-2vrzr\") pod \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") "
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.397406    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f921d6df-789f-4dca-8a1e-14c3d443fbc7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921d6df-789f-4dca-8a1e-14c3d443fbc7-maas-api-service-ca\") pod \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\" (UID: \"f921d6df-789f-4dca-8a1e-14c3d443fbc7\") "
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.397849    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm4pn\" (UniqueName: \"kubernetes.io/projected/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-kube-api-access-cm4pn\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.398053    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.398178    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f921d6df-789f-4dca-8a1e-14c3d443fbc7-maas-api-service-ca" pod "f921d6df-789f-4dca-8a1e-14c3d443fbc7" (UID: "f921d6df-789f-4dca-8a1e-14c3d443fbc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.398188    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.398470    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f921d6df-789f-4dca-8a1e-14c3d443fbc7-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.399044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.398964    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.401293 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.401259    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f921d6df-789f-4dca-8a1e-14c3d443fbc7-kube-api-access-2vrzr" pod "f921d6df-789f-4dca-8a1e-14c3d443fbc7" (UID: "f921d6df-789f-4dca-8a1e-14c3d443fbc7"). InnerVolumeSpecName "kube-api-access-2vrzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:20.401293 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.401267    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.409200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.409173    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm4pn\" (UniqueName: \"kubernetes.io/projected/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-kube-api-access-cm4pn\") pod \"maas-api-revoke-sub-5fe17c55-hp2jn\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.417703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.417679    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:20.420409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.420376    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:20.499843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.499746    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0066ff78-850a-4603-8eaa-0b533e03f28d-kube-api-access-q8sk9\" (UniqueName: \"kubernetes.io/projected/0066ff78-850a-4603-8eaa-0b533e03f28d-kube-api-access-q8sk9\") pod \"0066ff78-850a-4603-8eaa-0b533e03f28d\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") "
Oct 30 21:30:20.499843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.499826    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0066ff78-850a-4603-8eaa-0b533e03f28d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ff78-850a-4603-8eaa-0b533e03f28d-maas-api-service-ca\") pod \"0066ff78-850a-4603-8eaa-0b533e03f28d\" (UID: \"0066ff78-850a-4603-8eaa-0b533e03f28d\") "
Oct 30 21:30:20.500868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.500084    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40028a96-aedb-4c5a-9651-f1784ba3f52f-kube-api-access-f7944\" (UniqueName: \"kubernetes.io/projected/40028a96-aedb-4c5a-9651-f1784ba3f52f-kube-api-access-f7944\") pod \"40028a96-aedb-4c5a-9651-f1784ba3f52f\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") "
Oct 30 21:30:20.500868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.500186    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40028a96-aedb-4c5a-9651-f1784ba3f52f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40028a96-aedb-4c5a-9651-f1784ba3f52f-maas-api-service-ca\") pod \"40028a96-aedb-4c5a-9651-f1784ba3f52f\" (UID: \"40028a96-aedb-4c5a-9651-f1784ba3f52f\") "
Oct 30 21:30:20.500868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.500495    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0066ff78-850a-4603-8eaa-0b533e03f28d-maas-api-service-ca" pod "0066ff78-850a-4603-8eaa-0b533e03f28d" (UID: "0066ff78-850a-4603-8eaa-0b533e03f28d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:20.500868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.500814    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40028a96-aedb-4c5a-9651-f1784ba3f52f-maas-api-service-ca" pod "40028a96-aedb-4c5a-9651-f1784ba3f52f" (UID: "40028a96-aedb-4c5a-9651-f1784ba3f52f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:20.501119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.500934    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0066ff78-850a-4603-8eaa-0b533e03f28d-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.501119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.500960    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vrzr\" (UniqueName: \"kubernetes.io/projected/f921d6df-789f-4dca-8a1e-14c3d443fbc7-kube-api-access-2vrzr\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.502994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.502967    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0066ff78-850a-4603-8eaa-0b533e03f28d-kube-api-access-q8sk9" pod "0066ff78-850a-4603-8eaa-0b533e03f28d" (UID: "0066ff78-850a-4603-8eaa-0b533e03f28d"). InnerVolumeSpecName "kube-api-access-q8sk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:20.503225 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.503203    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40028a96-aedb-4c5a-9651-f1784ba3f52f-kube-api-access-f7944" pod "40028a96-aedb-4c5a-9651-f1784ba3f52f" (UID: "40028a96-aedb-4c5a-9651-f1784ba3f52f"). InnerVolumeSpecName "kube-api-access-f7944". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:20.524462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.524433    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:20.604183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.604150    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40028a96-aedb-4c5a-9651-f1784ba3f52f-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.604326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.604229    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8sk9\" (UniqueName: \"kubernetes.io/projected/0066ff78-850a-4603-8eaa-0b533e03f28d-kube-api-access-q8sk9\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.604326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.604250    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7944\" (UniqueName: \"kubernetes.io/projected/40028a96-aedb-4c5a-9651-f1784ba3f52f-kube-api-access-f7944\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.605931 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.605808    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:20.713592 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.712532    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:20.830152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.830124    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"]
Oct 30 21:30:20.834180 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:20.834142    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e54db2_6a16_44e6_a3a0_6b388c371acd.slice/crio-45c762945cb74129fb8e9635e21b70230cec5a6a150291fba208ce613e296e61 WatchSource:0}: Error finding container 45c762945cb74129fb8e9635e21b70230cec5a6a150291fba208ce613e296e61: Status 404 returned error can't find the container with id 45c762945cb74129fb8e9635e21b70230cec5a6a150291fba208ce613e296e61
Oct 30 21:30:20.936578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:20.936544    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"]
Oct 30 21:30:20.937645 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:30:20.937617    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1d1d8f_fbec_42a6_ac66_4fa53c72ec7e.slice/crio-031be109179dd9ce266ecccc92a8add7186b00050a718c46a37dfd66f422378c WatchSource:0}: Error finding container 031be109179dd9ce266ecccc92a8add7186b00050a718c46a37dfd66f422378c: Status 404 returned error can't find the container with id 031be109179dd9ce266ecccc92a8add7186b00050a718c46a37dfd66f422378c
Oct 30 21:30:21.045688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.045648    4852 generic.go:356] "Generic (PLEG): container finished" podID="cec79507-2620-4011-9ee9-04547f6810de" containerID="ed98f72f408834385ae984f69e5707a993fa9043d5da87b064504122d03afc66" exitCode=60
Oct 30 21:30:21.045688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.045677    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerDied","Data":"ed98f72f408834385ae984f69e5707a993fa9043d5da87b064504122d03afc66"}
Oct 30 21:30:21.045945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.045737    4852 scope.go:122] "RemoveContainer" containerID="4b1575aac4129013238061f94fa8e8713ea57fa3df9c3b8d6c8278f27ab855a5"
Oct 30 21:30:21.046195 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.046175    4852 scope.go:122] "RemoveContainer" containerID="ed98f72f408834385ae984f69e5707a993fa9043d5da87b064504122d03afc66"
Oct 30 21:30:21.046612 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:21.046583    4852 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92_odh-ai-gateway-infra(cec79507-2620-4011-9ee9-04547f6810de)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" podUID="cec79507-2620-4011-9ee9-04547f6810de"
Oct 30 21:30:21.048125 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.048099    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerStarted","Data":"84939aad816215ed9d7cd0b48a86f7739bf461d94234b616810ae0e658f5a0fd"}
Oct 30 21:30:21.053651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.053623    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn" event={"ID":"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e","Type":"ContainerStarted","Data":"031be109179dd9ce266ecccc92a8add7186b00050a718c46a37dfd66f422378c"}
Oct 30 21:30:21.056438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.055467    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt" event={"ID":"0066ff78-850a-4603-8eaa-0b533e03f28d","Type":"ContainerDied","Data":"5706e478c0ecfb6c349de4ecda4741b592411fb9f8400a65d238ae8b2db9c28f"}
Oct 30 21:30:21.056438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.055491    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"
Oct 30 21:30:21.059207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.059172    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn" event={"ID":"72e54db2-6a16-44e6-a3a0-6b388c371acd","Type":"ContainerStarted","Data":"45c762945cb74129fb8e9635e21b70230cec5a6a150291fba208ce613e296e61"}
Oct 30 21:30:21.068346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.068306    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"
Oct 30 21:30:21.068549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.068349    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w" event={"ID":"40028a96-aedb-4c5a-9651-f1784ba3f52f","Type":"ContainerDied","Data":"3d5d01604124ac8ce5c044ef77c7330f45d5fca03684c51ddb0495ca7777db5b"}
Oct 30 21:30:21.077615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.074166    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"
Oct 30 21:30:21.077615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.075656    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p" event={"ID":"f921d6df-789f-4dca-8a1e-14c3d443fbc7","Type":"ContainerDied","Data":"981a0542345964643b5a72ba3f8dce29294803a45679bf3c819f78c17ce58059"}
Oct 30 21:30:21.077615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.077241    4852 scope.go:122] "RemoveContainer" containerID="94bcf5e30a425fa874e205664435890093accbfe5ca124e19ad683b1c903771f"
Oct 30 21:30:21.101058 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.100966    4852 scope.go:122] "RemoveContainer" containerID="fde64e3a20d257c40ddef95b93a6255bd89726cd8b7f844e34df6d80e16529a4"
Oct 30 21:30:21.117443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.116132    4852 scope.go:122] "RemoveContainer" containerID="92e2273d1441cad57aa50e00380d32b6e46e258abc830dea8859dc1e342b7171"
Oct 30 21:30:21.132650 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.132619    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"]
Oct 30 21:30:21.163996 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.140651    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"]
Oct 30 21:30:21.163996 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.147922    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-ddcf93-29889930-mvx2w"]
Oct 30 21:30:21.173689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.172990    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"]
Oct 30 21:30:21.182442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.181490    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-e8063c-29889930-bt69p"]
Oct 30 21:30:21.195761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.195703    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"]
Oct 30 21:30:21.203866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.203837    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f147de-29889930-p9ckt"]
Oct 30 21:30:21.397783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.397758    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:21.402361 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.401803    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:21.434729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.434208    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57bcee80-0a44-4c57-bc04-19616abc74ff-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bcee80-0a44-4c57-bc04-19616abc74ff-maas-api-service-ca\") pod \"57bcee80-0a44-4c57-bc04-19616abc74ff\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") "
Oct 30 21:30:21.434729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.434257    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-xks-ca\") pod \"c27704f4-9744-46b0-9f3f-eb655a517c21\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") "
Oct 30 21:30:21.434729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.434335    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c27704f4-9744-46b0-9f3f-eb655a517c21-kube-api-access-5jgxw\" (UniqueName: \"kubernetes.io/projected/c27704f4-9744-46b0-9f3f-eb655a517c21-kube-api-access-5jgxw\") pod \"c27704f4-9744-46b0-9f3f-eb655a517c21\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") "
Oct 30 21:30:21.434729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.434487    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-openshift-ca\") pod \"c27704f4-9744-46b0-9f3f-eb655a517c21\" (UID: \"c27704f4-9744-46b0-9f3f-eb655a517c21\") "
Oct 30 21:30:21.434729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.434532    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57bcee80-0a44-4c57-bc04-19616abc74ff-kube-api-access-pfqns\" (UniqueName: \"kubernetes.io/projected/57bcee80-0a44-4c57-bc04-19616abc74ff-kube-api-access-pfqns\") pod \"57bcee80-0a44-4c57-bc04-19616abc74ff\" (UID: \"57bcee80-0a44-4c57-bc04-19616abc74ff\") "
Oct 30 21:30:21.435933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.435895    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-openshift-ca" pod "c27704f4-9744-46b0-9f3f-eb655a517c21" (UID: "c27704f4-9744-46b0-9f3f-eb655a517c21"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:21.435933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.435910    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57bcee80-0a44-4c57-bc04-19616abc74ff-maas-api-service-ca" pod "57bcee80-0a44-4c57-bc04-19616abc74ff" (UID: "57bcee80-0a44-4c57-bc04-19616abc74ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:21.438752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.438726    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57bcee80-0a44-4c57-bc04-19616abc74ff-kube-api-access-pfqns" pod "57bcee80-0a44-4c57-bc04-19616abc74ff" (UID: "57bcee80-0a44-4c57-bc04-19616abc74ff"). InnerVolumeSpecName "kube-api-access-pfqns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:21.438906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.438886    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c27704f4-9744-46b0-9f3f-eb655a517c21-kube-api-access-5jgxw" pod "c27704f4-9744-46b0-9f3f-eb655a517c21" (UID: "c27704f4-9744-46b0-9f3f-eb655a517c21"). InnerVolumeSpecName "kube-api-access-5jgxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:21.440211 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.440186    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-xks-ca" pod "c27704f4-9744-46b0-9f3f-eb655a517c21" (UID: "c27704f4-9744-46b0-9f3f-eb655a517c21"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:21.537205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.537170    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bcee80-0a44-4c57-bc04-19616abc74ff-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:21.537205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.537203    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:21.537409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.537220    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jgxw\" (UniqueName: \"kubernetes.io/projected/c27704f4-9744-46b0-9f3f-eb655a517c21-kube-api-access-5jgxw\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:21.537409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.537235    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c27704f4-9744-46b0-9f3f-eb655a517c21-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:21.537409 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.537248    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfqns\" (UniqueName: \"kubernetes.io/projected/57bcee80-0a44-4c57-bc04-19616abc74ff-kube-api-access-pfqns\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:21.560447 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:21.560373    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:30:21.614022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:21.613928    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:22.007257 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.007226    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" path="/var/lib/kubelet/pods/0066ff78-850a-4603-8eaa-0b533e03f28d/volumes"
Oct 30 21:30:22.007790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.007774    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="323e2d2f-adce-4489-81f6-159d46f03848" path="/var/lib/kubelet/pods/323e2d2f-adce-4489-81f6-159d46f03848/volumes"
Oct 30 21:30:22.008183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.008168    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" path="/var/lib/kubelet/pods/40028a96-aedb-4c5a-9651-f1784ba3f52f/volumes"
Oct 30 21:30:22.008809 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.008791    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" path="/var/lib/kubelet/pods/f921d6df-789f-4dca-8a1e-14c3d443fbc7/volumes"
Oct 30 21:30:22.119952 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.119916    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"]
Oct 30 21:30:22.125509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.125473    4852 generic.go:356] "Generic (PLEG): container finished" podID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerID="84939aad816215ed9d7cd0b48a86f7739bf461d94234b616810ae0e658f5a0fd" exitCode=60
Oct 30 21:30:22.125722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.125598    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerDied","Data":"84939aad816215ed9d7cd0b48a86f7739bf461d94234b616810ae0e658f5a0fd"}
Oct 30 21:30:22.125722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.125652    4852 scope.go:122] "RemoveContainer" containerID="5c9d81d89c3e35b030638b6a598b311972775d2995abc8bd07acc51ec4161bb1"
Oct 30 21:30:22.129147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.129037    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn" event={"ID":"57bcee80-0a44-4c57-bc04-19616abc74ff","Type":"ContainerDied","Data":"09b6dbc724ab68df59d39b56773d97fb7dcf36207345770b1ed5be3c96c448b4"}
Oct 30 21:30:22.129261 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.129155    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"
Oct 30 21:30:22.136205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.136169    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn" event={"ID":"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e","Type":"ContainerStarted","Data":"a176392279ce076cc4471ea7e2a6b835322e1ce31850690eff638503b8347fbe"}
Oct 30 21:30:22.139866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.139839    4852 generic.go:356] "Generic (PLEG): container finished" podID="72e54db2-6a16-44e6-a3a0-6b388c371acd" containerID="504f997795e69d66fce6ec4e0ddc127d18387e783fff28b65622a123dd435f52" exitCode=0
Oct 30 21:30:22.140000 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.139959    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn" event={"ID":"72e54db2-6a16-44e6-a3a0-6b388c371acd","Type":"ContainerDied","Data":"504f997795e69d66fce6ec4e0ddc127d18387e783fff28b65622a123dd435f52"}
Oct 30 21:30:22.143877 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.143855    4852 generic.go:356] "Generic (PLEG): container finished" podID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerID="307698abde47f9535848da222d9c06005cbdb24f1c51ca3d6650c70f2f63b48e" exitCode=0
Oct 30 21:30:22.144015 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.143985    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" event={"ID":"0685cd7c-3d01-4213-97e7-2215faee05ac","Type":"ContainerDied","Data":"307698abde47f9535848da222d9c06005cbdb24f1c51ca3d6650c70f2f63b48e"}
Oct 30 21:30:22.146520 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.145731    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"
Oct 30 21:30:22.146520 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.145665    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk" event={"ID":"c27704f4-9744-46b0-9f3f-eb655a517c21","Type":"ContainerDied","Data":"d04c7e4157c5b50115497ff72139a9dd98efd2d05fff71539a8b80197e074a27"}
Oct 30 21:30:22.146520 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.146005    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d04c7e4157c5b50115497ff72139a9dd98efd2d05fff71539a8b80197e074a27"
Oct 30 21:30:22.154559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.154535    4852 scope.go:122] "RemoveContainer" containerID="d7981bb461f07a95d1e86af3512242557b3687f03506e96d7eae0c347712cc4d"
Oct 30 21:30:22.201400 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.201348    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"]
Oct 30 21:30:22.221747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.221712    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6741b1-29889930-46xzn"]
Oct 30 21:30:22.242755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.230281    4852 scope.go:122] "RemoveContainer" containerID="310f4bc2693e524c60cd5a2632615c9ee6a283e4dc96cea1c7d83272cf0228a6"
Oct 30 21:30:22.242755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.234439    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:30:22.243132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.243066    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn" podStartSLOduration=2.243054489 podStartE2EDuration="2.243054489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:22.242491373 +0000 UTC m=+3130.357225566" watchObservedRunningTime="2026-10-30 21:30:22.243054489 +0000 UTC m=+3130.357788679"
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.352919    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-kube-api-access-p46gt\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-kube-api-access-p46gt\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353017    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0685cd7c-3d01-4213-97e7-2215faee05ac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0685cd7c-3d01-4213-97e7-2215faee05ac-istiod-ca-cert\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353058    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-certs\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353148    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-socket\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353167    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-data\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353198    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-envoy\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353285    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-credential-socket\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353311    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-podinfo\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.353499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.353346    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-token\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-token\") pod \"0685cd7c-3d01-4213-97e7-2215faee05ac\" (UID: \"0685cd7c-3d01-4213-97e7-2215faee05ac\") "
Oct 30 21:30:22.354878 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.354851    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0685cd7c-3d01-4213-97e7-2215faee05ac-istiod-ca-cert" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:22.355137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.355029    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-credential-socket" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.355230 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.355045    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-data" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.355335 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.355247    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-certs" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.356476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.356453    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-socket" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.363817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.363784    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-envoy" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:22.366906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.366876    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-kube-api-access-p46gt" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "kube-api-access-p46gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:22.367088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.367036    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-podinfo" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:22.368118 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.368091    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-token" pod "0685cd7c-3d01-4213-97e7-2215faee05ac" (UID: "0685cd7c-3d01-4213-97e7-2215faee05ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:22.393002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.392973    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:22.400593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.400568    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:22.454593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.454555    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cec79507-2620-4011-9ee9-04547f6810de-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec79507-2620-4011-9ee9-04547f6810de-maas-api-service-ca\") pod \"cec79507-2620-4011-9ee9-04547f6810de\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") "
Oct 30 21:30:22.454796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.454687    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eebf6d42-2bac-44f1-bbd5-f7bc24981893-kube-api-access-m62rr\" (UniqueName: \"kubernetes.io/projected/eebf6d42-2bac-44f1-bbd5-f7bc24981893-kube-api-access-m62rr\") pod \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") "
Oct 30 21:30:22.454884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.454811    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cec79507-2620-4011-9ee9-04547f6810de-kube-api-access-67582\" (UniqueName: \"kubernetes.io/projected/cec79507-2620-4011-9ee9-04547f6810de-kube-api-access-67582\") pod \"cec79507-2620-4011-9ee9-04547f6810de\" (UID: \"cec79507-2620-4011-9ee9-04547f6810de\") "
Oct 30 21:30:22.454948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.454907    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eebf6d42-2bac-44f1-bbd5-f7bc24981893-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebf6d42-2bac-44f1-bbd5-f7bc24981893-maas-api-service-ca\") pod \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\" (UID: \"eebf6d42-2bac-44f1-bbd5-f7bc24981893\") "
Oct 30 21:30:22.455075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455043    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cec79507-2620-4011-9ee9-04547f6810de-maas-api-service-ca" pod "cec79507-2620-4011-9ee9-04547f6810de" (UID: "cec79507-2620-4011-9ee9-04547f6810de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:22.455569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455527    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebf6d42-2bac-44f1-bbd5-f7bc24981893-maas-api-service-ca" pod "eebf6d42-2bac-44f1-bbd5-f7bc24981893" (UID: "eebf6d42-2bac-44f1-bbd5-f7bc24981893"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:22.455859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455829    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.455859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455854    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455869    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455883    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455896    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebf6d42-2bac-44f1-bbd5-f7bc24981893-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455910    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0685cd7c-3d01-4213-97e7-2215faee05ac-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455923    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455938    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455951    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p46gt\" (UniqueName: \"kubernetes.io/projected/0685cd7c-3d01-4213-97e7-2215faee05ac-kube-api-access-p46gt\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455966    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec79507-2620-4011-9ee9-04547f6810de-maas-api-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.456062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.455980    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0685cd7c-3d01-4213-97e7-2215faee05ac-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.458051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.458005    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cec79507-2620-4011-9ee9-04547f6810de-kube-api-access-67582" pod "cec79507-2620-4011-9ee9-04547f6810de" (UID: "cec79507-2620-4011-9ee9-04547f6810de"). InnerVolumeSpecName "kube-api-access-67582". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:22.461605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.461561    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebf6d42-2bac-44f1-bbd5-f7bc24981893-kube-api-access-m62rr" pod "eebf6d42-2bac-44f1-bbd5-f7bc24981893" (UID: "eebf6d42-2bac-44f1-bbd5-f7bc24981893"). InnerVolumeSpecName "kube-api-access-m62rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:22.524001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.523959    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:22.557738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.557698    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67582\" (UniqueName: \"kubernetes.io/projected/cec79507-2620-4011-9ee9-04547f6810de-kube-api-access-67582\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.557738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:22.557732    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m62rr\" (UniqueName: \"kubernetes.io/projected/eebf6d42-2bac-44f1-bbd5-f7bc24981893-kube-api-access-m62rr\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:23.165933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.165904    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"
Oct 30 21:30:23.166115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.165930    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k" event={"ID":"0685cd7c-3d01-4213-97e7-2215faee05ac","Type":"ContainerDied","Data":"b9e7ad5b442cbd6d383c5abbf5b27d507897ac94cc5d6d01e0fc2d8bebcac338"}
Oct 30 21:30:23.166115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.165989    4852 scope.go:122] "RemoveContainer" containerID="307698abde47f9535848da222d9c06005cbdb24f1c51ca3d6650c70f2f63b48e"
Oct 30 21:30:23.167874 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.167836    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"
Oct 30 21:30:23.167874 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.167842    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92" event={"ID":"cec79507-2620-4011-9ee9-04547f6810de","Type":"ContainerDied","Data":"103c28a439bfc3117cd1ad4346435aff49001411781706c7dcb4c6f80cc836ee"}
Oct 30 21:30:23.170062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.169981    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g" event={"ID":"eebf6d42-2bac-44f1-bbd5-f7bc24981893","Type":"ContainerDied","Data":"188410df43da0ccb88717375c74ca1fa6f50dde7143b3354cbd7a47ade1e33d4"}
Oct 30 21:30:23.170062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.170004    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"
Oct 30 21:30:23.176689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.176665    4852 generic.go:356] "Generic (PLEG): container finished" podID="6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" containerID="a176392279ce076cc4471ea7e2a6b835322e1ce31850690eff638503b8347fbe" exitCode=0
Oct 30 21:30:23.177185 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.176725    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn" event={"ID":"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e","Type":"ContainerDied","Data":"a176392279ce076cc4471ea7e2a6b835322e1ce31850690eff638503b8347fbe"}
Oct 30 21:30:23.179703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.179680    4852 scope.go:122] "RemoveContainer" containerID="ed98f72f408834385ae984f69e5707a993fa9043d5da87b064504122d03afc66"
Oct 30 21:30:23.192200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.192171    4852 scope.go:122] "RemoveContainer" containerID="84939aad816215ed9d7cd0b48a86f7739bf461d94234b616810ae0e658f5a0fd"
Oct 30 21:30:23.216140 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.216105    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"]
Oct 30 21:30:23.220987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.220956    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a3fb70-29889930-45d92"]
Oct 30 21:30:23.235440 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.234260    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"]
Oct 30 21:30:23.240177 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.240151    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-ffce13-openshift-default-745589f55f-jtl8k"]
Oct 30 21:30:23.253429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.253399    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"]
Oct 30 21:30:23.262974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.262941    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-f9a819-29889930-jtp9g"]
Oct 30 21:30:23.354997 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.354968    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:23.355171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.355148    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:23.470653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.470564    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72e54db2-6a16-44e6-a3a0-6b388c371acd-kube-api-access-znr4w\" (UniqueName: \"kubernetes.io/projected/72e54db2-6a16-44e6-a3a0-6b388c371acd-kube-api-access-znr4w\") pod \"72e54db2-6a16-44e6-a3a0-6b388c371acd\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") "
Oct 30 21:30:23.470653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.470615    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-openshift-ca\") pod \"72e54db2-6a16-44e6-a3a0-6b388c371acd\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") "
Oct 30 21:30:23.471076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.470872    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-xks-ca\") pod \"72e54db2-6a16-44e6-a3a0-6b388c371acd\" (UID: \"72e54db2-6a16-44e6-a3a0-6b388c371acd\") "
Oct 30 21:30:23.471076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.471064    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-openshift-ca" pod "72e54db2-6a16-44e6-a3a0-6b388c371acd" (UID: "72e54db2-6a16-44e6-a3a0-6b388c371acd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:23.471423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.471381    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:23.472863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.472839    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-xks-ca" pod "72e54db2-6a16-44e6-a3a0-6b388c371acd" (UID: "72e54db2-6a16-44e6-a3a0-6b388c371acd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:23.472954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.472886    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e54db2-6a16-44e6-a3a0-6b388c371acd-kube-api-access-znr4w" pod "72e54db2-6a16-44e6-a3a0-6b388c371acd" (UID: "72e54db2-6a16-44e6-a3a0-6b388c371acd"). InnerVolumeSpecName "kube-api-access-znr4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:23.573273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.573228    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/72e54db2-6a16-44e6-a3a0-6b388c371acd-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:23.573273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:23.573262    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znr4w\" (UniqueName: \"kubernetes.io/projected/72e54db2-6a16-44e6-a3a0-6b388c371acd-kube-api-access-znr4w\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:24.009255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.009209    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" path="/var/lib/kubelet/pods/0685cd7c-3d01-4213-97e7-2215faee05ac/volumes"
Oct 30 21:30:24.009873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.009847    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" path="/var/lib/kubelet/pods/57bcee80-0a44-4c57-bc04-19616abc74ff/volumes"
Oct 30 21:30:24.010359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.010336    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cec79507-2620-4011-9ee9-04547f6810de" path="/var/lib/kubelet/pods/cec79507-2620-4011-9ee9-04547f6810de/volumes"
Oct 30 21:30:24.011154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.011131    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" path="/var/lib/kubelet/pods/eebf6d42-2bac-44f1-bbd5-f7bc24981893/volumes"
Oct 30 21:30:24.119481 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.119443    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:24.241442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.241400    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn" event={"ID":"72e54db2-6a16-44e6-a3a0-6b388c371acd","Type":"ContainerDied","Data":"45c762945cb74129fb8e9635e21b70230cec5a6a150291fba208ce613e296e61"}
Oct 30 21:30:24.241719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.241474    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45c762945cb74129fb8e9635e21b70230cec5a6a150291fba208ce613e296e61"
Oct 30 21:30:24.241719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.241422    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"
Oct 30 21:30:24.436524 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.436489    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:24.511436 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.511373    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-kube-api-access-cm4pn\" (UniqueName: \"kubernetes.io/projected/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-kube-api-access-cm4pn\") pod \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") "
Oct 30 21:30:24.511837 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.511476    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-openshift-ca\") pod \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") "
Oct 30 21:30:24.511837 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.511733    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-xks-ca\") pod \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\" (UID: \"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e\") "
Oct 30 21:30:24.511958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.511865    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-openshift-ca" pod "6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" (UID: "6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:24.515452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.512425    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:24.516617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.516266    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-xks-ca" pod "6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" (UID: "6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:24.516617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.516445    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-kube-api-access-cm4pn" pod "6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" (UID: "6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e"). InnerVolumeSpecName "kube-api-access-cm4pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:24.613736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.613647    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm4pn\" (UniqueName: \"kubernetes.io/projected/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-kube-api-access-cm4pn\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:24.613736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:24.613690    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:25.032605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:25.032564    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:25.253458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:25.253427    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"
Oct 30 21:30:25.253651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:25.253426    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn" event={"ID":"6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e","Type":"ContainerDied","Data":"031be109179dd9ce266ecccc92a8add7186b00050a718c46a37dfd66f422378c"}
Oct 30 21:30:25.253651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:25.253556    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="031be109179dd9ce266ecccc92a8add7186b00050a718c46a37dfd66f422378c"
Oct 30 21:30:25.933551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:25.933513    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:26.633697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:26.633659    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:27.343962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:27.343924    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:27.841983 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:27.841944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:27.942307 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:27.942271    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:28.189491 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:28.170945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:28.866024 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:28.865986    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:29.540610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:29.540568    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:30.353132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:30.353083    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:30.450535 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:30.450498    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:30.877233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:30.877203    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:31.413637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:31.413604    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:32.345346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:32.345310    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.033044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:33.032992    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.729526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:33.729488    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:34.524836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:34.524804    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:35.335651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:35.335613    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:36.322774 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:36.322740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:37.170353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:37.170318    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:37.746278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:37.746240    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:37.961438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:37.961377    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:38.744706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:38.744670    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:39.582602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:39.582564    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:39.758986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:39.758949    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"]
Oct 30 21:30:39.759286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:39.759238    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv" podUID="7894ad8b-11d2-480f-92f1-8d18a3c2ba62" containerName="maas-api" containerID="cri-o://7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85" gracePeriod=30
Oct 30 21:30:39.806263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:39.806225    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:39.892832 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:39.892800    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"]
Oct 30 21:30:39.901736 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:39.901702    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9b78e4-pmtkk"]
Oct 30 21:30:39.904561 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:39.904526    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.191146 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.191042    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c27704f4-9744-46b0-9f3f-eb655a517c21" path="/var/lib/kubelet/pods/c27704f4-9744-46b0-9f3f-eb655a517c21/volumes"
Oct 30 21:30:40.240812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.240779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.247302 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.247276    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:30:40.293448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.291133    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"]
Oct 30 21:30:40.295796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.295768    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a3fb70-m54qn"]
Oct 30 21:30:40.337738 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.337701    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.362584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.362540    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-metrics-certs\") pod \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") "
Oct 30 21:30:40.362769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.362705    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-openshift-service-ca\") pod \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") "
Oct 30 21:30:40.362829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.362798    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-kube-api-access-tfhb2\" (UniqueName: \"kubernetes.io/projected/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-kube-api-access-tfhb2\") pod \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") "
Oct 30 21:30:40.362883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.362872    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls\") pod \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\" (UID: \"7894ad8b-11d2-480f-92f1-8d18a3c2ba62\") "
Oct 30 21:30:40.363078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.363052    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-openshift-service-ca" pod "7894ad8b-11d2-480f-92f1-8d18a3c2ba62" (UID: "7894ad8b-11d2-480f-92f1-8d18a3c2ba62"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:40.363478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.363445    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:40.365011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.364984    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-kube-api-access-tfhb2" pod "7894ad8b-11d2-480f-92f1-8d18a3c2ba62" (UID: "7894ad8b-11d2-480f-92f1-8d18a3c2ba62"). InnerVolumeSpecName "kube-api-access-tfhb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:40.365242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.365210    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-metrics-certs" pod "7894ad8b-11d2-480f-92f1-8d18a3c2ba62" (UID: "7894ad8b-11d2-480f-92f1-8d18a3c2ba62"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:40.365335 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.365224    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls" pod "7894ad8b-11d2-480f-92f1-8d18a3c2ba62" (UID: "7894ad8b-11d2-480f-92f1-8d18a3c2ba62"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:40.429691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.429656    4852 generic.go:356] "Generic (PLEG): container finished" podID="7894ad8b-11d2-480f-92f1-8d18a3c2ba62" containerID="7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85" exitCode=0
Oct 30 21:30:40.429899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.429775    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv" event={"ID":"7894ad8b-11d2-480f-92f1-8d18a3c2ba62","Type":"ContainerDied","Data":"7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85"}
Oct 30 21:30:40.429899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.429815    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv" event={"ID":"7894ad8b-11d2-480f-92f1-8d18a3c2ba62","Type":"ContainerDied","Data":"b06f136084639eb54312e924b45a59034b2ff6df50bdf705ad19543a48440677"}
Oct 30 21:30:40.429899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.429848    4852 scope.go:122] "RemoveContainer" containerID="7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85"
Oct 30 21:30:40.430647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.429998    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"
Oct 30 21:30:40.438672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.438642    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.443372 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.443349    4852 scope.go:122] "RemoveContainer" containerID="7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85"
Oct 30 21:30:40.443688 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:40.443668    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85\": container with ID starting with 7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85 not found: ID does not exist" containerID="7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85"
Oct 30 21:30:40.443755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.443701    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85"} err="failed to get container status \"7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85\": rpc error: code = NotFound desc = could not find container \"7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85\": container with ID starting with 7426324a9f9d5c6050d9623babfa085e7c31184e992618e045270ee278c48f85 not found: ID does not exist"
Oct 30 21:30:40.456899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.456864    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"]
Oct 30 21:30:40.461303 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.461274    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a3fb70-9b596676b-2zcgv"]
Oct 30 21:30:40.465299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.465268    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:40.465423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.465301    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:40.465423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.465319    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfhb2\" (UniqueName: \"kubernetes.io/projected/7894ad8b-11d2-480f-92f1-8d18a3c2ba62-kube-api-access-tfhb2\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:40.793926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:40.793868    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:41.135349 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:41.135269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:41.322719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:41.322674    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:41.418975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:41.418881    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:41.984157 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:41.984117    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:42.006848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:42.006811    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72e54db2-6a16-44e6-a3a0-6b388c371acd" path="/var/lib/kubelet/pods/72e54db2-6a16-44e6-a3a0-6b388c371acd/volumes"
Oct 30 21:30:42.007287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:42.007273    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7894ad8b-11d2-480f-92f1-8d18a3c2ba62" path="/var/lib/kubelet/pods/7894ad8b-11d2-480f-92f1-8d18a3c2ba62/volumes"
Oct 30 21:30:42.744879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:42.744851    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:43.384511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:43.384472    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:43.551999 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:43.550419    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:43.673066 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:43.672958    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:43.772121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:43.772085    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.175306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:44.175268    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.522180 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:44.522146    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:45.446357 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:45.446301    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:46.321577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:46.321539    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:47.141703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:47.141665    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:47.246665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:47.246591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:47.465927 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:47.465845    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"]
Oct 30 21:30:47.466333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:47.466110    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" containerID="cri-o://e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19" gracePeriod=30
Oct 30 21:30:47.560758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:47.560659    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:48.131360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:48.131329    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:48.234526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:48.234494    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:48.334191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:48.334155    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:48.532125 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:48.532093    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"]
Oct 30 21:30:48.532626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:48.532309    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" containerID="cri-o://649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68" gracePeriod=30
Oct 30 21:30:48.558457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:48.558357    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:49.248057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:49.248018    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:50.111804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:50.111768    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:50.802219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:50.802185    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:50.940884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:50.940851    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:51.042703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:51.042659    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:51.068700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:51.068619    4852 patch_prober.go:29] interesting pod/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:30:51.068865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:51.068691    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:30:51.283935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:51.283896    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:51.559304 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:51.559260    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:30:51.781668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:51.781633    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:52.711631 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.711597    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:52.763572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.763154    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:30:52.921761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.921722    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istiod-ca-cert\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.921946 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.921807    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-envoy\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.921946 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.921871    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-credential-socket\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.921946 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.921917    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-data\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.922127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.921945    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-certs\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.922127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922028    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-kube-api-access-rwg48\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-kube-api-access-rwg48\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.922127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922101    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-podinfo\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.922290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922130    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-socket\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.922290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922127    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istiod-ca-cert" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:52.922290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922247    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-data" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:52.922290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922264    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-credential-socket" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:52.922290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922277    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-token\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-token\") pod \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\" (UID: \"5d9690d9-dfb2-47c7-805e-0a84c8a89b42\") "
Oct 30 21:30:52.922290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922285    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-certs" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:52.922642 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922486    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-socket" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:52.922973 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922943    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:52.922973 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922975    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:52.923170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.922990    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:52.923170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.923004    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:52.923170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.923019    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:52.924249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.924219    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-envoy" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:52.924352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.924242    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-podinfo" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:52.924415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.924354    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-token" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:52.924694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:52.924665    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-kube-api-access-rwg48" pod "5d9690d9-dfb2-47c7-805e-0a84c8a89b42" (UID: "5d9690d9-dfb2-47c7-805e-0a84c8a89b42"). InnerVolumeSpecName "kube-api-access-rwg48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:53.024500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.024466    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.024500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.024495    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.024500 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.024506    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwg48\" (UniqueName: \"kubernetes.io/projected/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-kube-api-access-rwg48\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.024735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.024516    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d9690d9-dfb2-47c7-805e-0a84c8a89b42-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.391888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.391796    4852 patch_prober.go:29] interesting pod/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:30:53.391888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.391870    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:30:53.535949 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.535911    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:53.543612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.543583    4852 generic.go:356] "Generic (PLEG): container finished" podID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerID="e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19" exitCode=0
Oct 30 21:30:53.543804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.543640    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"
Oct 30 21:30:53.543804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.543661    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" event={"ID":"5d9690d9-dfb2-47c7-805e-0a84c8a89b42","Type":"ContainerDied","Data":"e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19"}
Oct 30 21:30:53.543804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.543716    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw" event={"ID":"5d9690d9-dfb2-47c7-805e-0a84c8a89b42","Type":"ContainerDied","Data":"5d993eda9c2e67e388aca3f37c4ead7bd5a4ea6dd6ebf3627709a8c3a16094ec"}
Oct 30 21:30:53.543804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.543739    4852 scope.go:122] "RemoveContainer" containerID="e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19"
Oct 30 21:30:53.557305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.557275    4852 scope.go:122] "RemoveContainer" containerID="437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af"
Oct 30 21:30:53.571772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.571743    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"]
Oct 30 21:30:53.577476 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.577452    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-9b78e4-openshift-default-64bbf7f759-x72sw"]
Oct 30 21:30:53.647335 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.647186    4852 scope.go:122] "RemoveContainer" containerID="e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19"
Oct 30 21:30:53.647599 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:53.647574    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19\": container with ID starting with e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19 not found: ID does not exist" containerID="e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19"
Oct 30 21:30:53.647656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.647610    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19"} err="failed to get container status \"e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19\": rpc error: code = NotFound desc = could not find container \"e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19\": container with ID starting with e88958a4206204dd6b1aec330da4a56ca3c71d2b52b6f0edf74fa3835072bd19 not found: ID does not exist"
Oct 30 21:30:53.647656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.647632    4852 scope.go:122] "RemoveContainer" containerID="437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af"
Oct 30 21:30:53.647981 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:53.647958    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af\": container with ID starting with 437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af not found: ID does not exist" containerID="437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af"
Oct 30 21:30:53.648063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.647991    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af"} err="failed to get container status \"437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af\": rpc error: code = NotFound desc = could not find container \"437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af\": container with ID starting with 437480b6930d948394d147560b43e568deda5510f1461033a02ddedcb59616af not found: ID does not exist"
Oct 30 21:30:53.746104 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.746055    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"]
Oct 30 21:30:53.747260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.747217    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy" containerID="cri-o://4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181" gracePeriod=30
Oct 30 21:30:53.823563 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.823538    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:30:53.939320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939241    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-socket\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939302    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-data\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939339    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-credential-socket\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939533    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-envoy\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939563    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istiod-ca-cert\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939665    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-token\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-token\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939626    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-socket" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:53.939755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939689    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-credential-socket" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:53.939755 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939719    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-certs\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939968 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939769    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-kube-api-access-spzvp\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-kube-api-access-spzvp\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.939968 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939837    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-podinfo\") pod \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\" (UID: \"0ec5fb08-ff18-48b2-9611-eee1690dbb07\") "
Oct 30 21:30:53.940083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939997    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-certs" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:53.940197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.940174    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istiod-ca-cert" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:53.940587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.940561    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.940587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.940590    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.940798 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.940606    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.940798 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.940620    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:53.940798 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.939640    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-data" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:53.942081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.942054    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-envoy" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:53.942197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.942092    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-token" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:53.942285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.942268    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-podinfo" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:53.942703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:53.942674    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-kube-api-access-spzvp" pod "0ec5fb08-ff18-48b2-9611-eee1690dbb07" (UID: "0ec5fb08-ff18-48b2-9611-eee1690dbb07"). InnerVolumeSpecName "kube-api-access-spzvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:54.006239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.006204    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" path="/var/lib/kubelet/pods/5d9690d9-dfb2-47c7-805e-0a84c8a89b42/volumes"
Oct 30 21:30:54.041986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.041951    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:54.041986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.041981    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spzvp\" (UniqueName: \"kubernetes.io/projected/0ec5fb08-ff18-48b2-9611-eee1690dbb07-kube-api-access-spzvp\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:54.041986 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.041991    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:54.042237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.042000    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:54.042237 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.042008    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ec5fb08-ff18-48b2-9611-eee1690dbb07-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:54.328360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.328319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:54.554080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.554047    4852 generic.go:356] "Generic (PLEG): container finished" podID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerID="649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68" exitCode=0
Oct 30 21:30:54.554280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.554106    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"
Oct 30 21:30:54.554280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.554115    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" event={"ID":"0ec5fb08-ff18-48b2-9611-eee1690dbb07","Type":"ContainerDied","Data":"649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68"}
Oct 30 21:30:54.554280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.554202    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx" event={"ID":"0ec5fb08-ff18-48b2-9611-eee1690dbb07","Type":"ContainerDied","Data":"f74d739f14ee2400efc2278a6a3df8c8d1fbc492cbfa7c017d771509953bb5ab"}
Oct 30 21:30:54.554280 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.554226    4852 scope.go:122] "RemoveContainer" containerID="649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68"
Oct 30 21:30:54.565870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.565848    4852 scope.go:122] "RemoveContainer" containerID="649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68"
Oct 30 21:30:54.566161 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:54.566144    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68\": container with ID starting with 649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68 not found: ID does not exist" containerID="649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68"
Oct 30 21:30:54.566221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.566168    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68"} err="failed to get container status \"649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68\": rpc error: code = NotFound desc = could not find container \"649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68\": container with ID starting with 649577f5587380cc99b31c1a8c9f3a8cc6000f5f835e1cea4098bdcb71750a68 not found: ID does not exist"
Oct 30 21:30:54.612472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.612352    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"]
Oct 30 21:30:54.633598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:54.633562    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a3fb70-openshift-default-98f99985f-wkhxx"]
Oct 30 21:30:55.117588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:55.117523    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:55.912707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:55.912676    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:56.019921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:56.019885    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" path="/var/lib/kubelet/pods/0ec5fb08-ff18-48b2-9611-eee1690dbb07/volumes"
Oct 30 21:30:56.626752 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:56.626720    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:57.432262 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:57.432226    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:58.114542 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:58.114491    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:58.226120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:58.226083    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:58.325674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:58.325643    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:58.962322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:58.962263    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:59.072544 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.072468    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:30:59.166169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166125    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-data\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166222    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-socket\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166257    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-envoy\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166311    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istiod-ca-cert\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166363    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-podinfo\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166423    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-certs\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166442    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-data" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166531    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-token\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-token\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166570    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-socket" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:59.166678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166616    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-credential-socket\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.167269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166705    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-kube-api-access-spmx7\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-kube-api-access-spmx7\") pod \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\" (UID: \"564948ce-6be6-44e8-97b5-04b7c1e1ac80\") "
Oct 30 21:30:59.167269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166728    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-certs" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:59.167269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166844    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istiod-ca-cert" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:59.167269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.166993    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-credential-socket" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:59.167549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.167332    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.167549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.167357    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.167549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.167374    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.167549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.167423    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.167549 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.167439    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.169006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.168981    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-kube-api-access-spmx7" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "kube-api-access-spmx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:59.169192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.169173    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-podinfo" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:30:59.169467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.169441    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-envoy" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:59.170060 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.170035    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-token" pod "564948ce-6be6-44e8-97b5-04b7c1e1ac80" (UID: "564948ce-6be6-44e8-97b5-04b7c1e1ac80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:59.269344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.269302    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.269344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.269348    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.269608 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.269363    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.269608 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.269378    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spmx7\" (UniqueName: \"kubernetes.io/projected/564948ce-6be6-44e8-97b5-04b7c1e1ac80-kube-api-access-spmx7\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:59.292665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.292628    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:59.393012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.392916    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:59.619416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.618507    4852 generic.go:356] "Generic (PLEG): container finished" podID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerID="4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181" exitCode=0
Oct 30 21:30:59.619416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.618680    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" event={"ID":"564948ce-6be6-44e8-97b5-04b7c1e1ac80","Type":"ContainerDied","Data":"4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181"}
Oct 30 21:30:59.619416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.618742    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd" event={"ID":"564948ce-6be6-44e8-97b5-04b7c1e1ac80","Type":"ContainerDied","Data":"293133539ac3c8da57421ae70f7c08ad5067a81d3f6b0b5bf2adb6a555318eea"}
Oct 30 21:30:59.619416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.618773    4852 scope.go:122] "RemoveContainer" containerID="4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181"
Oct 30 21:30:59.619416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.618932    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"
Oct 30 21:30:59.646358 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.646335    4852 scope.go:122] "RemoveContainer" containerID="4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181"
Oct 30 21:30:59.646744 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:30:59.646720    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181\": container with ID starting with 4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181 not found: ID does not exist" containerID="4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181"
Oct 30 21:30:59.646843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.646753    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181"} err="failed to get container status \"4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181\": rpc error: code = NotFound desc = could not find container \"4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181\": container with ID starting with 4c3f488323db2db2e3524e9e75dcdc5093481c2e4a2df91ab2d78d5171bbd181 not found: ID does not exist"
Oct 30 21:30:59.655085 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.655052    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"]
Oct 30 21:30:59.661320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.661289    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-e07442-openshift-default-56d77688d5-bjsvd"]
Oct 30 21:30:59.839901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:30:59.839869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:00.006153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:00.006125    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" path="/var/lib/kubelet/pods/564948ce-6be6-44e8-97b5-04b7c1e1ac80/volumes"
Oct 30 21:31:00.627020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:00.626973    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.315221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:01.315187    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.502373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:01.502303    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.752344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:01.752255    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.851933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:01.851895    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:02.024505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:02.024468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:02.124760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:02.124731    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:02.242819 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:02.242764    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:02.749205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:02.749168    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:03.559043 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:03.559009    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:04.216224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:04.216184    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:06.555641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:06.555605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:06.636573 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:06.636479    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:07.110803 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:07.110772    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:07.333371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:07.333328    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:07.421849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:07.421757    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:07.519423 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:07.519367    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:07.911889 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:07.911849    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:08.382902 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:08.382865    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:09.249611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:09.249578    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:09.717068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:09.717034    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:09.818716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:09.818670    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:10.054460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:10.054382    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:10.154505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:10.154468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:10.910165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:10.910133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:11.733589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:11.733559    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:12.530537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:12.530503    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:13.326851 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:13.326801    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:13.504121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:13.504052    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:13.605787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:13.605691    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:13.847014 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:13.846982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:14.248900 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:14.248869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:14.356191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:14.356144    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:14.634577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:14.634545    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"]
Oct 30 21:31:14.635031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:14.634832    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy" containerID="cri-o://0fd184959dfd1fbf24abaa5c129ed4d07240d2befb51c3efb4bc4043f11e729c" gracePeriod=30
Oct 30 21:31:14.672769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:14.672740    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:15.319653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:15.319618    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:16.304985 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:16.304941    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:17.015612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:17.015580    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:17.795767 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:17.795727    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:18.522033 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:18.521996    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:19.247019 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:19.246982    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:19.872274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:19.872242    4852 generic.go:356] "Generic (PLEG): container finished" podID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerID="0fd184959dfd1fbf24abaa5c129ed4d07240d2befb51c3efb4bc4043f11e729c" exitCode=0
Oct 30 21:31:19.872496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:19.872296    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" event={"ID":"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d","Type":"ContainerDied","Data":"0fd184959dfd1fbf24abaa5c129ed4d07240d2befb51c3efb4bc4043f11e729c"}
Oct 30 21:31:19.872496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:19.872364    4852 scope.go:122] "RemoveContainer" containerID="d67730157f6a28cedacf449392389f300e00ed697f042edbaa6864e2336df756"
Oct 30 21:31:19.920939 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:19.920881    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:31:20.046561 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046520    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-credential-socket\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.046770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046572    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-token\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-token\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.046770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046652    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-data\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.046770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046695    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-kube-api-access-xlnlh\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-kube-api-access-xlnlh\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.046960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046852    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-credential-socket" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:20.046960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046862    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-certs\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.046960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046921    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-socket\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.047148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.046994    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-envoy\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.047148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047024    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-data" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:20.047148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047088    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-certs" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:20.047148 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047125    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-podinfo\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.047366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047165    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istiod-ca-cert\") pod \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\" (UID: \"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d\") "
Oct 30 21:31:20.047366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047196    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-socket" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:20.047603 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047544    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istiod-ca-cert" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:31:20.047788 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047770    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.047860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047795    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.047860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047809    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.047860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047824    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.047860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.047837    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.049538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.049458    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-envoy" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:20.049538 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.049525    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-podinfo" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:31:20.049695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.049572    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-token" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:20.049695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.049621    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-kube-api-access-xlnlh" pod "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" (UID: "b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d"). InnerVolumeSpecName "kube-api-access-xlnlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:20.120617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.120574    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:20.148993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.148958    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.148993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.148992    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.149215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.149007    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.149215 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.149022    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlnlh\" (UniqueName: \"kubernetes.io/projected/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d-kube-api-access-xlnlh\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:20.729771 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.729723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:20.832364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.832334    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:20.881953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.881918    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"
Oct 30 21:31:20.881953 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.881923    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr" event={"ID":"b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d","Type":"ContainerDied","Data":"c196bbe6e8c8b25d385c483928903abd5f97d3b9ff2cf85c595d91f84a7c6104"}
Oct 30 21:31:20.882224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.881979    4852 scope.go:122] "RemoveContainer" containerID="0fd184959dfd1fbf24abaa5c129ed4d07240d2befb51c3efb4bc4043f11e729c"
Oct 30 21:31:20.907919 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.907890    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"]
Oct 30 21:31:20.915112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:20.915079    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5506f4-openshift-default-75fcfff45c-5cpnr"]
Oct 30 21:31:21.154231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:21.154195    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:21.559987 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:21.559953    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:31:22.005376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:22.005302    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" path="/var/lib/kubelet/pods/b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d/volumes"
Oct 30 21:31:22.026061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:22.026033    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:22.825119 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:22.825072    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:23.421445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:23.421409    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:23.637413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:23.636546    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:23.761553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:23.760330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:23.853691 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:23.853652    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:24.263102 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:24.263018    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:24.533473 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:24.531209    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:24.621915 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:24.621878    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:24.714786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:24.714745    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:25.385981 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:25.385944    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"]
Oct 30 21:31:25.445267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:25.445230    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:25.546319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:25.546291    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:25.646250 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:25.646164    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:26.364415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:26.364341    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:27.116470 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:27.116431    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:27.829347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:27.829308    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.650950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:28.650906    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"]
Oct 30 21:31:28.651941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:28.651876    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv" podUID="3a0d7c61-76d4-4830-9181-5875507551f3" containerName="maas-api" containerID="cri-o://7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204" gracePeriod=30
Oct 30 21:31:28.658555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:28.658526    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.795106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:28.795065    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.874709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:28.874646    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.952263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:28.952236    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.954096 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:28.954078    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:31:29.133599 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.133558    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a0d7c61-76d4-4830-9181-5875507551f3-kube-api-access-mrbvf\" (UniqueName: \"kubernetes.io/projected/3a0d7c61-76d4-4830-9181-5875507551f3-kube-api-access-mrbvf\") pod \"3a0d7c61-76d4-4830-9181-5875507551f3\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") "
Oct 30 21:31:29.133820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.133670    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-metrics-certs\") pod \"3a0d7c61-76d4-4830-9181-5875507551f3\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") "
Oct 30 21:31:29.133820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.133756    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls\") pod \"3a0d7c61-76d4-4830-9181-5875507551f3\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") "
Oct 30 21:31:29.134106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.134085    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a0d7c61-76d4-4830-9181-5875507551f3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d7c61-76d4-4830-9181-5875507551f3-openshift-service-ca\") pod \"3a0d7c61-76d4-4830-9181-5875507551f3\" (UID: \"3a0d7c61-76d4-4830-9181-5875507551f3\") "
Oct 30 21:31:29.136056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.136011    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a0d7c61-76d4-4830-9181-5875507551f3-openshift-service-ca" pod "3a0d7c61-76d4-4830-9181-5875507551f3" (UID: "3a0d7c61-76d4-4830-9181-5875507551f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:31:29.158049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.157995    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-metrics-certs" pod "3a0d7c61-76d4-4830-9181-5875507551f3" (UID: "3a0d7c61-76d4-4830-9181-5875507551f3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:31:29.158721 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.158668    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls" pod "3a0d7c61-76d4-4830-9181-5875507551f3" (UID: "3a0d7c61-76d4-4830-9181-5875507551f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:31:29.166628 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.166582    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a0d7c61-76d4-4830-9181-5875507551f3-kube-api-access-mrbvf" pod "3a0d7c61-76d4-4830-9181-5875507551f3" (UID: "3a0d7c61-76d4-4830-9181-5875507551f3"). InnerVolumeSpecName "kube-api-access-mrbvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:29.169352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.168959    4852 generic.go:356] "Generic (PLEG): container finished" podID="3a0d7c61-76d4-4830-9181-5875507551f3" containerID="7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204" exitCode=0
Oct 30 21:31:29.169352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.169040    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"
Oct 30 21:31:29.169352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.169188    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv" event={"ID":"3a0d7c61-76d4-4830-9181-5875507551f3","Type":"ContainerDied","Data":"7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204"}
Oct 30 21:31:29.169352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.169231    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv" event={"ID":"3a0d7c61-76d4-4830-9181-5875507551f3","Type":"ContainerDied","Data":"3bc3d72c0e65d8f76b21ba41202dc0a7f81cb1cfdb2c1abc7cd29460dd8592fc"}
Oct 30 21:31:29.169352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.169251    4852 scope.go:122] "RemoveContainer" containerID="7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204"
Oct 30 21:31:29.187672 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.187648    4852 scope.go:122] "RemoveContainer" containerID="7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204"
Oct 30 21:31:29.188069 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:29.188045    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204\": container with ID starting with 7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204 not found: ID does not exist" containerID="7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204"
Oct 30 21:31:29.188191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.188082    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204"} err="failed to get container status \"7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204\": rpc error: code = NotFound desc = could not find container \"7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204\": container with ID starting with 7350d8261e2eeb61c6535b0f7399a8c96dbc416ccb59f0dd46f9c6ef7782a204 not found: ID does not exist"
Oct 30 21:31:29.200115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.200080    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:29.203063 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.202968    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"]
Oct 30 21:31:29.208892 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.208864    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-00909f-9685df98d-f6fsv"]
Oct 30 21:31:29.236699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.236648    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrbvf\" (UniqueName: \"kubernetes.io/projected/3a0d7c61-76d4-4830-9181-5875507551f3-kube-api-access-mrbvf\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:29.236699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.236702    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:29.236901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.236718    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a0d7c61-76d4-4830-9181-5875507551f3-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:29.236901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.236731    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a0d7c61-76d4-4830-9181-5875507551f3-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:29.304116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.304084    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:29.495990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.495901    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:29.765333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:29.765302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:30.007651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:30.007614    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a0d7c61-76d4-4830-9181-5875507551f3" path="/var/lib/kubelet/pods/3a0d7c61-76d4-4830-9181-5875507551f3/volumes"
Oct 30 21:31:30.223227 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:30.223142    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:30.386709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:30.386662    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" podUID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" containerName="payload-processing" containerID="cri-o://45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c" gracePeriod=25
Oct 30 21:31:31.122804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:31.122765    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:31.263213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:31.263173    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" podUID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:31:31.980555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:31.980511    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:32.085374 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:32.085340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:32.177419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:32.177361    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:32.394084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:32.394043    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"]
Oct 30 21:31:32.394415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:32.394359    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy" containerID="cri-o://9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6" gracePeriod=30
Oct 30 21:31:32.421616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:32.421586    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:32.849647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:32.849603    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:33.814641 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:33.814601    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.628003 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:34.627964    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:35.465976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:35.465944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:36.262064 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:36.262031    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.038514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.038456    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.470029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.469988    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.572757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.572717    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.693048 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.693022    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:31:37.724999 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.724959    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.774626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.774592    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-certs\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.774850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.774662    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-data\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.774850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.774764    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-credential-socket\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.774850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.774806    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-envoy\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.775023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.774865    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istiod-ca-cert\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.775023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.774991    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-podinfo\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.775132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.775007    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-data" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:37.775132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.775048    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-token\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-token\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.775132 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.775078    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-kube-api-access-2lhmf\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-kube-api-access-2lhmf\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.775290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.775187    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-socket\") pod \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\" (UID: \"5f57cfad-f8d4-4b39-a024-dfefeba9b431\") "
Oct 30 21:31:37.775994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.775753    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.776158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.776000    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-socket" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:37.777110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.776485    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-credential-socket" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:37.777110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.776852    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istiod-ca-cert" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:31:37.777110 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.777106    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-certs" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:37.781467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.779441    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-kube-api-access-2lhmf" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "kube-api-access-2lhmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:37.781467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.780283    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-envoy" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:37.782090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.782057    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-token" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:37.782305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.782283    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-podinfo" pod "5f57cfad-f8d4-4b39-a024-dfefeba9b431" (UID: "5f57cfad-f8d4-4b39-a024-dfefeba9b431"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:31:37.795683 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.795656    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"]
Oct 30 21:31:37.795922 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.795898    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" containerID="cri-o://f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0" gracePeriod=30
Oct 30 21:31:37.823143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.823029    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.877564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877527    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.877564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877560    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.877564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877571    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.877829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877585    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.877829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877597    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.877829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877606    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lhmf\" (UniqueName: \"kubernetes.io/projected/5f57cfad-f8d4-4b39-a024-dfefeba9b431-kube-api-access-2lhmf\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.877829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877618    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.877829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.877627    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f57cfad-f8d4-4b39-a024-dfefeba9b431-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:37.922492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:37.922373    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:38.099820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.044516    4852 patch_prober.go:29] interesting pod/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:31:38.099820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.044585    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:31:38.099820 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.090251    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:38.270645 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.270611    4852 generic.go:356] "Generic (PLEG): container finished" podID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerID="9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6" exitCode=0
Oct 30 21:31:38.270933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.270660    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"
Oct 30 21:31:38.270933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.270672    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" event={"ID":"5f57cfad-f8d4-4b39-a024-dfefeba9b431","Type":"ContainerDied","Data":"9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6"}
Oct 30 21:31:38.270933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.270730    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn" event={"ID":"5f57cfad-f8d4-4b39-a024-dfefeba9b431","Type":"ContainerDied","Data":"35b825fe5988d8a5e0a45fabeb1d615079fdc98faad879b3ad0c1cb2c1f1e03a"}
Oct 30 21:31:38.270933 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.270760    4852 scope.go:122] "RemoveContainer" containerID="9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6"
Oct 30 21:31:38.284169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.284144    4852 scope.go:122] "RemoveContainer" containerID="1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f"
Oct 30 21:31:38.295617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.295594    4852 scope.go:122] "RemoveContainer" containerID="9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6"
Oct 30 21:31:38.295920 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:38.295896    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6\": container with ID starting with 9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6 not found: ID does not exist" containerID="9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6"
Oct 30 21:31:38.295975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.295932    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6"} err="failed to get container status \"9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6\": rpc error: code = NotFound desc = could not find container \"9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6\": container with ID starting with 9bc5d3424b92c81ece65a80452acf1eeada1a7cafd558caeba4635c5ea5b49d6 not found: ID does not exist"
Oct 30 21:31:38.295975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.295955    4852 scope.go:122] "RemoveContainer" containerID="1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f"
Oct 30 21:31:38.296204 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:38.296186    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f\": container with ID starting with 1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f not found: ID does not exist" containerID="1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f"
Oct 30 21:31:38.296251 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.296211    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f"} err="failed to get container status \"1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f\": rpc error: code = NotFound desc = could not find container \"1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f\": container with ID starting with 1b636f9716aa3f52c78394b34c9ee2fda584a74468b069c156be9e62d7e8ff6f not found: ID does not exist"
Oct 30 21:31:38.298562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.298541    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"]
Oct 30 21:31:38.307260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.307228    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-e1e0f5-openshift-default-dd58f9dcc-slkjn"]
Oct 30 21:31:38.922845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:38.922806    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:39.184836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:39.184732    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:39.932152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:39.932119    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:40.016057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:40.016019    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" path="/var/lib/kubelet/pods/5f57cfad-f8d4-4b39-a024-dfefeba9b431/volumes"
Oct 30 21:31:40.752696 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:40.752665    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:41.262922 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:41.262881    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" podUID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:31:41.531377 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:41.531287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:42.384722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:42.384672    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:43.067969 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.067896    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:31:43.068131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.068090    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"]
Oct 30 21:31:43.081613 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.081579    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"]
Oct 30 21:31:43.132373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.132337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:43.162614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.162578    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/969d8145-ba12-4e33-837e-5f13d0d71291-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969d8145-ba12-4e33-837e-5f13d0d71291-istiod-ca-cert\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.162796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.162669    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-credential-socket\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.162796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.162704    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-envoy\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.162796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.162778    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-kube-api-access-dmqgg\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-kube-api-access-dmqgg\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.163010 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.162864    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/969d8145-ba12-4e33-837e-5f13d0d71291-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969d8145-ba12-4e33-837e-5f13d0d71291-istio-podinfo\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.163010 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.162914    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-istio-token\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-istio-token\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.163010 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.162961    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-certs\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.163178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.163041    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-data\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.163178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.163047    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969d8145-ba12-4e33-837e-5f13d0d71291-istiod-ca-cert" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:31:43.163178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.163091    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-socket\") pod \"969d8145-ba12-4e33-837e-5f13d0d71291\" (UID: \"969d8145-ba12-4e33-837e-5f13d0d71291\") "
Oct 30 21:31:43.163367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.163346    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-credential-socket" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:43.163592 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.163570    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-socket" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:43.163852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.163828    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-certs" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:43.163966 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.163948    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-data" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:43.165272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.164285    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.165272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.164316    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.165272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.164330    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.165272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.164343    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.165272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.164357    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969d8145-ba12-4e33-837e-5f13d0d71291-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.167025 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.166998    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-kube-api-access-dmqgg" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "kube-api-access-dmqgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:43.167888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.167849    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-envoy" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:43.168021 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.167992    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/969d8145-ba12-4e33-837e-5f13d0d71291-istio-podinfo" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:31:43.168165 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.168138    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-istio-token" pod "969d8145-ba12-4e33-837e-5f13d0d71291" (UID: "969d8145-ba12-4e33-837e-5f13d0d71291"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:43.233616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.233582    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:43.265862 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.265820    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969d8145-ba12-4e33-837e-5f13d0d71291-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.265862 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.265857    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmqgg\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-kube-api-access-dmqgg\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.266072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.265874    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969d8145-ba12-4e33-837e-5f13d0d71291-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.266072 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.265888    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969d8145-ba12-4e33-837e-5f13d0d71291-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:43.319001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.318615    4852 generic.go:356] "Generic (PLEG): container finished" podID="969d8145-ba12-4e33-837e-5f13d0d71291" containerID="f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0" exitCode=0
Oct 30 21:31:43.319001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.318670    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"
Oct 30 21:31:43.319001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.318665    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" event={"ID":"969d8145-ba12-4e33-837e-5f13d0d71291","Type":"ContainerDied","Data":"f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0"}
Oct 30 21:31:43.319001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.318788    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g" event={"ID":"969d8145-ba12-4e33-837e-5f13d0d71291","Type":"ContainerDied","Data":"e5e6c1d0b223df369367446e85db60cd4212363ce8cdda22420f4a0b318b7820"}
Oct 30 21:31:43.319001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.318824    4852 scope.go:122] "RemoveContainer" containerID="f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0"
Oct 30 21:31:43.332411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.332371    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:43.333692 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.333675    4852 scope.go:122] "RemoveContainer" containerID="f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0"
Oct 30 21:31:43.334021 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:43.334001    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0\": container with ID starting with f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0 not found: ID does not exist" containerID="f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0"
Oct 30 21:31:43.334079 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.334029    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0"} err="failed to get container status \"f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0\": rpc error: code = NotFound desc = could not find container \"f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0\": container with ID starting with f298caad66a361ff0fd97000a75fd863b959251385421a453475489744f095a0 not found: ID does not exist"
Oct 30 21:31:43.345632 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.345605    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"]
Oct 30 21:31:43.352775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:43.352750    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-00909f-openshift-default-7c565998f7-h2t6g"]
Oct 30 21:31:44.005351 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:44.005314    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" path="/var/lib/kubelet/pods/969d8145-ba12-4e33-837e-5f13d0d71291/volumes"
Oct 30 21:31:44.112724 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:44.112690    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:45.051917 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:45.051888    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:46.128009 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:46.127980    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:46.913693 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:46.913658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:47.502729 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:47.502697    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:47.703411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:47.698367    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:47.795529 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:47.795496    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:47.898410 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:47.897968    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.068507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.068378    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" podUID="bd7151ac-50e1-411b-a52f-f65a6e246b94" containerName="payload-processing" containerID="cri-o://18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca" gracePeriod=25
Oct 30 21:31:48.082919 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.082883    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" podUID="c6d83db6-f575-405f-b80b-4c65738bfb48" containerName="payload-pre-processing" containerID="cri-o://335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122" gracePeriod=25
Oct 30 21:31:48.294703 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.294666    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.448166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.448078    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"]
Oct 30 21:31:48.511697 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.511667    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.610923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.610892    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.763976 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.763922    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.768606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.768576    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"]
Oct 30 21:31:48.867783 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:48.867746    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:49.844825 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:49.844791    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:50.266286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.266254    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:50.668780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.668747    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:31:50.795944 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.795905    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e636851-f7e8-46d2-bd43-f7202a0fcb32-kube-api-access-p7m7h\" (UniqueName: \"kubernetes.io/projected/4e636851-f7e8-46d2-bd43-f7202a0fcb32-kube-api-access-p7m7h\") pod \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") "
Oct 30 21:31:50.796233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.796033    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e636851-f7e8-46d2-bd43-f7202a0fcb32-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e636851-f7e8-46d2-bd43-f7202a0fcb32-plugins-config-volume\") pod \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\" (UID: \"4e636851-f7e8-46d2-bd43-f7202a0fcb32\") "
Oct 30 21:31:50.796536 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.796493    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e636851-f7e8-46d2-bd43-f7202a0fcb32-plugins-config-volume" pod "4e636851-f7e8-46d2-bd43-f7202a0fcb32" (UID: "4e636851-f7e8-46d2-bd43-f7202a0fcb32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:31:50.796761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.796737    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e636851-f7e8-46d2-bd43-f7202a0fcb32-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:50.798222 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.798194    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e636851-f7e8-46d2-bd43-f7202a0fcb32-kube-api-access-p7m7h" pod "4e636851-f7e8-46d2-bd43-f7202a0fcb32" (UID: "4e636851-f7e8-46d2-bd43-f7202a0fcb32"). InnerVolumeSpecName "kube-api-access-p7m7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:50.814768 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.814709    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:50.898327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:50.898289    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7m7h\" (UniqueName: \"kubernetes.io/projected/4e636851-f7e8-46d2-bd43-f7202a0fcb32-kube-api-access-p7m7h\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:51.413572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.413536    4852 generic.go:356] "Generic (PLEG): container finished" podID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" containerID="45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c" exitCode=0
Oct 30 21:31:51.413786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.413589    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"
Oct 30 21:31:51.413786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.413622    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" event={"ID":"4e636851-f7e8-46d2-bd43-f7202a0fcb32","Type":"ContainerDied","Data":"45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c"}
Oct 30 21:31:51.413786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.413680    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss" event={"ID":"4e636851-f7e8-46d2-bd43-f7202a0fcb32","Type":"ContainerDied","Data":"a7f8ece0b759bd713312f6e0ffb2a22b3fc107c34f5ff0b612df1334691173fe"}
Oct 30 21:31:51.413786 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.413704    4852 scope.go:122] "RemoveContainer" containerID="45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c"
Oct 30 21:31:51.425176 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.425154    4852 scope.go:122] "RemoveContainer" containerID="45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c"
Oct 30 21:31:51.425553 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:51.425530    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c\": container with ID starting with 45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c not found: ID does not exist" containerID="45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c"
Oct 30 21:31:51.425644 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.425561    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c"} err="failed to get container status \"45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c\": rpc error: code = NotFound desc = could not find container \"45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c\": container with ID starting with 45bf23f4dedee3a6744075047de622ae33fc5caa36e413aaaf9fdb673606c56c not found: ID does not exist"
Oct 30 21:31:51.444275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.444247    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"]
Oct 30 21:31:51.448992 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.448967    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-00909f-64695dc5f5-mjrss"]
Oct 30 21:31:51.561297 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:51.561261    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:31:51.650116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:51.650083    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:52.006680 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:52.006648    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" path="/var/lib/kubelet/pods/4e636851-f7e8-46d2-bd43-f7202a0fcb32/volumes"
Oct 30 21:31:52.438169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:52.438075    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:53.362924 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.362890    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:53.470443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.459170    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2" podUID="311556e8-83a5-4fbe-a7ec-5efcb6d0432f" containerName="payload-processing" containerID="cri-o://c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c" gracePeriod=25
Oct 30 21:31:53.745667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.745637    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:31:53.769239 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.769198    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" podUID="6a24bc29-1110-4746-9bca-5be86231f077" containerName="payload-pre-processing" containerID="cri-o://63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214" gracePeriod=25
Oct 30 21:31:53.837109 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.837072    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-plugins-config-volume\") pod \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") "
Oct 30 21:31:53.837366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.837220    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-kube-api-access-dbzfr\" (UniqueName: \"kubernetes.io/projected/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-kube-api-access-dbzfr\") pod \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\" (UID: \"311556e8-83a5-4fbe-a7ec-5efcb6d0432f\") "
Oct 30 21:31:53.838248 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.838213    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-plugins-config-volume" pod "311556e8-83a5-4fbe-a7ec-5efcb6d0432f" (UID: "311556e8-83a5-4fbe-a7ec-5efcb6d0432f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:31:53.838715 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.838684    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:53.840483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.840454    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-kube-api-access-dbzfr" pod "311556e8-83a5-4fbe-a7ec-5efcb6d0432f" (UID: "311556e8-83a5-4fbe-a7ec-5efcb6d0432f"). InnerVolumeSpecName "kube-api-access-dbzfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:53.940863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:53.940784    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbzfr\" (UniqueName: \"kubernetes.io/projected/311556e8-83a5-4fbe-a7ec-5efcb6d0432f-kube-api-access-dbzfr\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:54.081767 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.081723    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:54.475370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.475331    4852 generic.go:356] "Generic (PLEG): container finished" podID="311556e8-83a5-4fbe-a7ec-5efcb6d0432f" containerID="c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c" exitCode=0
Oct 30 21:31:54.475903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.475426    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"
Oct 30 21:31:54.475903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.475421    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2" event={"ID":"311556e8-83a5-4fbe-a7ec-5efcb6d0432f","Type":"ContainerDied","Data":"c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c"}
Oct 30 21:31:54.475903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.475555    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2" event={"ID":"311556e8-83a5-4fbe-a7ec-5efcb6d0432f","Type":"ContainerDied","Data":"dcb42ca71a761cd546056d0209fa38c3083a498cb854cb87d872f0e52dd93695"}
Oct 30 21:31:54.475903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.475576    4852 scope.go:122] "RemoveContainer" containerID="c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c"
Oct 30 21:31:54.487296 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.487265    4852 scope.go:122] "RemoveContainer" containerID="c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c"
Oct 30 21:31:54.487621 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:31:54.487598    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c\": container with ID starting with c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c not found: ID does not exist" containerID="c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c"
Oct 30 21:31:54.487713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.487628    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c"} err="failed to get container status \"c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c\": rpc error: code = NotFound desc = could not find container \"c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c\": container with ID starting with c69209b205f86c086fbaf5261888783d07361dfc1665015180798cae451ea65c not found: ID does not exist"
Oct 30 21:31:54.498923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.498886    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"]
Oct 30 21:31:54.502463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.502434    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f147de-659b45bcff-4mht2"]
Oct 30 21:31:54.730730 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.730630    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" podUID="bd7151ac-50e1-411b-a52f-f65a6e246b94" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:31:54.733181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:54.733149    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" podUID="c6d83db6-f575-405f-b80b-4c65738bfb48" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:31:55.107807 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:55.107769    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:55.486360 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:55.486272    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:55.589218 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:55.589179    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:55.792362 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:55.792326    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"]
Oct 30 21:31:55.792884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:55.792852    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy" containerID="cri-o://8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2" gracePeriod=30
Oct 30 21:31:55.816197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:55.816162    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:55.924735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:55.924699    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:56.042445 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:56.042261    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="311556e8-83a5-4fbe-a7ec-5efcb6d0432f" path="/var/lib/kubelet/pods/311556e8-83a5-4fbe-a7ec-5efcb6d0432f/volumes"
Oct 30 21:31:56.647399 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:56.647367    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:57.000331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:57.000252    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:57.098004 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:57.097965    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:57.191221 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:57.191182    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:57.684045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:57.684005    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:58.038960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:58.038923    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:58.813922 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:58.813889    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:59.017965 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:59.017929    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:59.128264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:59.128176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:59.248977 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:59.248945    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:59.588339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:31:59.588287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:00.123127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:00.123065    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:00.570127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:00.570092    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"]
Oct 30 21:32:00.638870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:00.638826    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:00.738246 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:00.738211    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:00.928116 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:00.927997    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:01.151089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.151061    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:32:01.198458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198427    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-kube-api-access-plk97\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-kube-api-access-plk97\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.198653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198531    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47aaf1c6-f339-46d3-b3d5-a65397293537-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47aaf1c6-f339-46d3-b3d5-a65397293537-istiod-ca-cert\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.198653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198588    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-podinfo\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.198653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198614    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-envoy\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.198653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198651    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-certs\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.198895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198734    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-token\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-token\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.198895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198827    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-data\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.198895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198853    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-socket\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.199062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198917    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-credential-socket\") pod \"47aaf1c6-f339-46d3-b3d5-a65397293537\" (UID: \"47aaf1c6-f339-46d3-b3d5-a65397293537\") "
Oct 30 21:32:01.199062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.198998    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47aaf1c6-f339-46d3-b3d5-a65397293537-istiod-ca-cert" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:01.199319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.199289    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-certs" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:01.199458 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.199431    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-socket" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:01.199534 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.199503    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-data" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:01.199605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.199542    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.199605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.199574    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-credential-socket" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:01.199605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.199582    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47aaf1c6-f339-46d3-b3d5-a65397293537-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.199782 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.199619    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.201203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.201174    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-kube-api-access-plk97" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "kube-api-access-plk97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:01.201371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.201344    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-envoy" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:01.201541 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.201420    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-podinfo" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:01.201816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.201796    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-token" pod "47aaf1c6-f339-46d3-b3d5-a65397293537" (UID: "47aaf1c6-f339-46d3-b3d5-a65397293537"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:01.300852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.300812    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.300852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.300845    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.300852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.300854    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.300852 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.300864    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.301156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.300873    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47aaf1c6-f339-46d3-b3d5-a65397293537-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.301156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.300881    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plk97\" (UniqueName: \"kubernetes.io/projected/47aaf1c6-f339-46d3-b3d5-a65397293537-kube-api-access-plk97\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:01.528589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.528556    4852 generic.go:356] "Generic (PLEG): container finished" podID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerID="8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2" exitCode=0
Oct 30 21:32:01.528799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.528613    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"
Oct 30 21:32:01.528799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.528635    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" event={"ID":"47aaf1c6-f339-46d3-b3d5-a65397293537","Type":"ContainerDied","Data":"8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2"}
Oct 30 21:32:01.528799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.528692    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv" event={"ID":"47aaf1c6-f339-46d3-b3d5-a65397293537","Type":"ContainerDied","Data":"9529ba7ee9089b7488aba60969ce661b31270b56b9007e24a03ef86d912bbcba"}
Oct 30 21:32:01.528799 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.528713    4852 scope.go:122] "RemoveContainer" containerID="8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2"
Oct 30 21:32:01.542884 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.542857    4852 scope.go:122] "RemoveContainer" containerID="d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824"
Oct 30 21:32:01.554161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.554047    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"]
Oct 30 21:32:01.555283 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.555267    4852 scope.go:122] "RemoveContainer" containerID="8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2"
Oct 30 21:32:01.555656 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:01.555620    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2\": container with ID starting with 8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2 not found: ID does not exist" containerID="8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2"
Oct 30 21:32:01.555766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.555655    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2"} err="failed to get container status \"8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2\": rpc error: code = NotFound desc = could not find container \"8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2\": container with ID starting with 8634c649d9016b40d4bc2c25900a0e1772ae11ffdbd6841854735008a48ad1c2 not found: ID does not exist"
Oct 30 21:32:01.555766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.555675    4852 scope.go:122] "RemoveContainer" containerID="d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824"
Oct 30 21:32:01.555987 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:01.555963    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824\": container with ID starting with d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824 not found: ID does not exist" containerID="d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824"
Oct 30 21:32:01.556055 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.555996    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824"} err="failed to get container status \"d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824\": rpc error: code = NotFound desc = could not find container \"d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824\": container with ID starting with d402096a21f995045b101ed6c346ebd62fb27daa69cda3e8e2cff303c3bdf824 not found: ID does not exist"
Oct 30 21:32:01.557973 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.557954    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-3a65ec-openshift-default-9f9996889-snwhv"]
Oct 30 21:32:01.694653 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:01.694623    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:02.004945 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.004865    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" path="/var/lib/kubelet/pods/47aaf1c6-f339-46d3-b3d5-a65397293537/volumes"
Oct 30 21:32:02.255412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.255298    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"]
Oct 30 21:32:02.255890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.255725    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5" podUID="b69450b8-9338-46e2-9c5a-094fad128ea4" containerName="maas-api" containerID="cri-o://42a5a1f5f3a150542aa9c41decc938905593300ee0254c781a683a44ed0132c8" gracePeriod=30
Oct 30 21:32:02.285601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.285523    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:02.465455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.461033    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:02.575192 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.574944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:02.605051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.605017    4852 generic.go:356] "Generic (PLEG): container finished" podID="b69450b8-9338-46e2-9c5a-094fad128ea4" containerID="42a5a1f5f3a150542aa9c41decc938905593300ee0254c781a683a44ed0132c8" exitCode=0
Oct 30 21:32:02.605232 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.605099    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5" event={"ID":"b69450b8-9338-46e2-9c5a-094fad128ea4","Type":"ContainerDied","Data":"42a5a1f5f3a150542aa9c41decc938905593300ee0254c781a683a44ed0132c8"}
Oct 30 21:32:02.654411 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.653465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:02.669039 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.657451    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:32:02.692796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.692761    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" podUID="6a24bc29-1110-4746-9bca-5be86231f077" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:02.757668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.757627    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-maas-api-tls\") pod \"b69450b8-9338-46e2-9c5a-094fad128ea4\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") "
Oct 30 21:32:02.757845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.757730    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b69450b8-9338-46e2-9c5a-094fad128ea4-kube-api-access-g4t7x\" (UniqueName: \"kubernetes.io/projected/b69450b8-9338-46e2-9c5a-094fad128ea4-kube-api-access-g4t7x\") pod \"b69450b8-9338-46e2-9c5a-094fad128ea4\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") "
Oct 30 21:32:02.757845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.757792    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b69450b8-9338-46e2-9c5a-094fad128ea4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69450b8-9338-46e2-9c5a-094fad128ea4-openshift-service-ca\") pod \"b69450b8-9338-46e2-9c5a-094fad128ea4\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") "
Oct 30 21:32:02.757928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.757874    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-metrics-certs\") pod \"b69450b8-9338-46e2-9c5a-094fad128ea4\" (UID: \"b69450b8-9338-46e2-9c5a-094fad128ea4\") "
Oct 30 21:32:02.758268 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.758237    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b69450b8-9338-46e2-9c5a-094fad128ea4-openshift-service-ca" pod "b69450b8-9338-46e2-9c5a-094fad128ea4" (UID: "b69450b8-9338-46e2-9c5a-094fad128ea4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:02.758449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.758376    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69450b8-9338-46e2-9c5a-094fad128ea4-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:02.760117 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.760087    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-maas-api-tls" pod "b69450b8-9338-46e2-9c5a-094fad128ea4" (UID: "b69450b8-9338-46e2-9c5a-094fad128ea4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:02.760223 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.760168    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-metrics-certs" pod "b69450b8-9338-46e2-9c5a-094fad128ea4" (UID: "b69450b8-9338-46e2-9c5a-094fad128ea4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:02.760317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.760299    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b69450b8-9338-46e2-9c5a-094fad128ea4-kube-api-access-g4t7x" pod "b69450b8-9338-46e2-9c5a-094fad128ea4" (UID: "b69450b8-9338-46e2-9c5a-094fad128ea4"). InnerVolumeSpecName "kube-api-access-g4t7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:02.859147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.859062    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:02.859147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.859091    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b69450b8-9338-46e2-9c5a-094fad128ea4-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:02.859147 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.859101    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4t7x\" (UniqueName: \"kubernetes.io/projected/b69450b8-9338-46e2-9c5a-094fad128ea4-kube-api-access-g4t7x\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:02.980331 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:02.980280    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:03.335454 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:03.335416    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:03.614044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:03.613966    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"
Oct 30 21:32:03.614044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:03.614005    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5" event={"ID":"b69450b8-9338-46e2-9c5a-094fad128ea4","Type":"ContainerDied","Data":"ebd5c8a57e775525c5676f6fbb259f56efc52e63732d3d10ff4d0520807634be"}
Oct 30 21:32:03.614306 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:03.614077    4852 scope.go:122] "RemoveContainer" containerID="42a5a1f5f3a150542aa9c41decc938905593300ee0254c781a683a44ed0132c8"
Oct 30 21:32:03.641438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:03.641398    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"]
Oct 30 21:32:03.646429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:03.646402    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0c4052-5d48b58884-kkqh5"]
Oct 30 21:32:04.004863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:04.004779    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b69450b8-9338-46e2-9c5a-094fad128ea4" path="/var/lib/kubelet/pods/b69450b8-9338-46e2-9c5a-094fad128ea4/volumes"
Oct 30 21:32:04.164275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:04.164235    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:04.730229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:04.730187    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" podUID="bd7151ac-50e1-411b-a52f-f65a6e246b94" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:04.733106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:04.733062    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" podUID="c6d83db6-f575-405f-b80b-4c65738bfb48" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:05.127955 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:05.127915    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:05.250709 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:05.250669    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:05.570664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:05.570610    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974" podUID="0335594b-ba42-4f48-96d0-3196c63389ad" containerName="payload-processing" containerID="cri-o://2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9" gracePeriod=25
Oct 30 21:32:05.572056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:05.572021    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"]
Oct 30 21:32:05.572572 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:05.572360    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy" containerID="cri-o://a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23" gracePeriod=30
Oct 30 21:32:05.594772 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:32:05.594745    4852 logging.go:55] [core] [Channel #10915 SubChannel #10916]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.181:50052", ServerName: "10.133.0.181:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.181:50052: connect: connection refused"
Oct 30 21:32:05.912457 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:05.912427    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:32:06.012987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.012956    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0335594b-ba42-4f48-96d0-3196c63389ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0335594b-ba42-4f48-96d0-3196c63389ad-plugins-config-volume\") pod \"0335594b-ba42-4f48-96d0-3196c63389ad\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") "
Oct 30 21:32:06.013182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.013068    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0335594b-ba42-4f48-96d0-3196c63389ad-kube-api-access-cp2xm\" (UniqueName: \"kubernetes.io/projected/0335594b-ba42-4f48-96d0-3196c63389ad-kube-api-access-cp2xm\") pod \"0335594b-ba42-4f48-96d0-3196c63389ad\" (UID: \"0335594b-ba42-4f48-96d0-3196c63389ad\") "
Oct 30 21:32:06.013406 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.013360    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0335594b-ba42-4f48-96d0-3196c63389ad-plugins-config-volume" pod "0335594b-ba42-4f48-96d0-3196c63389ad" (UID: "0335594b-ba42-4f48-96d0-3196c63389ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:06.013596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.013576    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0335594b-ba42-4f48-96d0-3196c63389ad-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:06.015346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.015315    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0335594b-ba42-4f48-96d0-3196c63389ad-kube-api-access-cp2xm" pod "0335594b-ba42-4f48-96d0-3196c63389ad" (UID: "0335594b-ba42-4f48-96d0-3196c63389ad"). InnerVolumeSpecName "kube-api-access-cp2xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:06.115485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.115449    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp2xm\" (UniqueName: \"kubernetes.io/projected/0335594b-ba42-4f48-96d0-3196c63389ad-kube-api-access-cp2xm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:06.319432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.317206    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:06.507438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.507399    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"]
Oct 30 21:32:06.526504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.526469    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"]
Oct 30 21:32:06.576265 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.576177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:06.595433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.595363    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974" podUID="0335594b-ba42-4f48-96d0-3196c63389ad" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.181:50052\" within 1s: context deadline exceeded"
Oct 30 21:32:06.595611 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:32:06.595415    4852 logging.go:55] [core] [Channel #10915 SubChannel #10916]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.181:50052", ServerName: "10.133.0.181:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.181:50052: operation was canceled"
Oct 30 21:32:06.643245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.643212    4852 generic.go:356] "Generic (PLEG): container finished" podID="0335594b-ba42-4f48-96d0-3196c63389ad" containerID="2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9" exitCode=0
Oct 30 21:32:06.643442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.643272    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"
Oct 30 21:32:06.643442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.643289    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974" event={"ID":"0335594b-ba42-4f48-96d0-3196c63389ad","Type":"ContainerDied","Data":"2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9"}
Oct 30 21:32:06.643442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.643344    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974" event={"ID":"0335594b-ba42-4f48-96d0-3196c63389ad","Type":"ContainerDied","Data":"0de8778623871d94bf9c0603db199d9bb55ba53caf16ace2fcc743250e395eea"}
Oct 30 21:32:06.643442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.643366    4852 scope.go:122] "RemoveContainer" containerID="2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9"
Oct 30 21:32:06.654928 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.654911    4852 scope.go:122] "RemoveContainer" containerID="2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9"
Oct 30 21:32:06.655226 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:06.655207    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9\": container with ID starting with 2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9 not found: ID does not exist" containerID="2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9"
Oct 30 21:32:06.655275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.655234    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9"} err="failed to get container status \"2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9\": rpc error: code = NotFound desc = could not find container \"2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9\": container with ID starting with 2db3b69127a57ceea773b86a9d84a233a54a9d5906967f59dbf43bb95386a4a9 not found: ID does not exist"
Oct 30 21:32:06.669588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.669556    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"]
Oct 30 21:32:06.671493 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.671468    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:06.675415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.675367    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0c4052-5fcb7d75df-pr974"]
Oct 30 21:32:06.955811 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:06.955726    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:07.397649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:07.397614    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:07.498495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:07.498463    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:07.734115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:07.734006    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:07.812899 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:07.812869    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:08.005159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.005078    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0335594b-ba42-4f48-96d0-3196c63389ad" path="/var/lib/kubelet/pods/0335594b-ba42-4f48-96d0-3196c63389ad/volumes"
Oct 30 21:32:08.239264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.239213    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:08.348516 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.348488    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:32:08.398896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.398867    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:32:08.456144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.455423    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6d83db6-f575-405f-b80b-4c65738bfb48-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6d83db6-f575-405f-b80b-4c65738bfb48-plugins-config-volume\") pod \"c6d83db6-f575-405f-b80b-4c65738bfb48\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") "
Oct 30 21:32:08.456144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.455634    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6d83db6-f575-405f-b80b-4c65738bfb48-kube-api-access-ws7j6\" (UniqueName: \"kubernetes.io/projected/c6d83db6-f575-405f-b80b-4c65738bfb48-kube-api-access-ws7j6\") pod \"c6d83db6-f575-405f-b80b-4c65738bfb48\" (UID: \"c6d83db6-f575-405f-b80b-4c65738bfb48\") "
Oct 30 21:32:08.456144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.455790    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd7151ac-50e1-411b-a52f-f65a6e246b94-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd7151ac-50e1-411b-a52f-f65a6e246b94-plugins-config-volume\") pod \"bd7151ac-50e1-411b-a52f-f65a6e246b94\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") "
Oct 30 21:32:08.456144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.455833    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6d83db6-f575-405f-b80b-4c65738bfb48-plugins-config-volume" pod "c6d83db6-f575-405f-b80b-4c65738bfb48" (UID: "c6d83db6-f575-405f-b80b-4c65738bfb48"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:08.456144 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.455859    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd7151ac-50e1-411b-a52f-f65a6e246b94-kube-api-access-glb6k\" (UniqueName: \"kubernetes.io/projected/bd7151ac-50e1-411b-a52f-f65a6e246b94-kube-api-access-glb6k\") pod \"bd7151ac-50e1-411b-a52f-f65a6e246b94\" (UID: \"bd7151ac-50e1-411b-a52f-f65a6e246b94\") "
Oct 30 21:32:08.458467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.456464    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6d83db6-f575-405f-b80b-4c65738bfb48-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:08.458467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.457074    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd7151ac-50e1-411b-a52f-f65a6e246b94-plugins-config-volume" pod "bd7151ac-50e1-411b-a52f-f65a6e246b94" (UID: "bd7151ac-50e1-411b-a52f-f65a6e246b94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:08.460111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.460081    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7151ac-50e1-411b-a52f-f65a6e246b94-kube-api-access-glb6k" pod "bd7151ac-50e1-411b-a52f-f65a6e246b94" (UID: "bd7151ac-50e1-411b-a52f-f65a6e246b94"). InnerVolumeSpecName "kube-api-access-glb6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:08.461634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.461608    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d83db6-f575-405f-b80b-4c65738bfb48-kube-api-access-ws7j6" pod "c6d83db6-f575-405f-b80b-4c65738bfb48" (UID: "c6d83db6-f575-405f-b80b-4c65738bfb48"). InnerVolumeSpecName "kube-api-access-ws7j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:08.557372 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.557285    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws7j6\" (UniqueName: \"kubernetes.io/projected/c6d83db6-f575-405f-b80b-4c65738bfb48-kube-api-access-ws7j6\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:08.557372 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.557319    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd7151ac-50e1-411b-a52f-f65a6e246b94-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:08.557372 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.557331    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glb6k\" (UniqueName: \"kubernetes.io/projected/bd7151ac-50e1-411b-a52f-f65a6e246b94-kube-api-access-glb6k\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:08.658551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.658514    4852 generic.go:356] "Generic (PLEG): container finished" podID="c6d83db6-f575-405f-b80b-4c65738bfb48" containerID="335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122" exitCode=0
Oct 30 21:32:08.658754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.658583    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"
Oct 30 21:32:08.658754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.658595    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" event={"ID":"c6d83db6-f575-405f-b80b-4c65738bfb48","Type":"ContainerDied","Data":"335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122"}
Oct 30 21:32:08.658754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.658676    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb" event={"ID":"c6d83db6-f575-405f-b80b-4c65738bfb48","Type":"ContainerDied","Data":"6d0c59facccf5bd261dfbcf2e50578b533fa2a66abebfaba073c3c172a96bbae"}
Oct 30 21:32:08.658754 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.658706    4852 scope.go:122] "RemoveContainer" containerID="335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122"
Oct 30 21:32:08.660219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.660190    4852 generic.go:356] "Generic (PLEG): container finished" podID="bd7151ac-50e1-411b-a52f-f65a6e246b94" containerID="18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca" exitCode=0
Oct 30 21:32:08.660338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.660255    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"
Oct 30 21:32:08.660338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.660286    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" event={"ID":"bd7151ac-50e1-411b-a52f-f65a6e246b94","Type":"ContainerDied","Data":"18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca"}
Oct 30 21:32:08.660465 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.660356    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk" event={"ID":"bd7151ac-50e1-411b-a52f-f65a6e246b94","Type":"ContainerDied","Data":"0b2e526e93e86a80d74ca08c6cc3675ba765e2b62ab2476577b2e07f71145917"}
Oct 30 21:32:08.669444 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.669346    4852 scope.go:122] "RemoveContainer" containerID="335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122"
Oct 30 21:32:08.669776 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:08.669754    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122\": container with ID starting with 335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122 not found: ID does not exist" containerID="335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122"
Oct 30 21:32:08.669861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.669784    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122"} err="failed to get container status \"335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122\": rpc error: code = NotFound desc = could not find container \"335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122\": container with ID starting with 335dd3310efc3caf4e4c6b68696e9ec0562a47c86d46417c23eaf20299f28122 not found: ID does not exist"
Oct 30 21:32:08.669861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.669802    4852 scope.go:122] "RemoveContainer" containerID="18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca"
Oct 30 21:32:08.681480 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.681460    4852 scope.go:122] "RemoveContainer" containerID="18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca"
Oct 30 21:32:08.681776 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:08.681756    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca\": container with ID starting with 18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca not found: ID does not exist" containerID="18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca"
Oct 30 21:32:08.681865 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.681783    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca"} err="failed to get container status \"18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca\": rpc error: code = NotFound desc = could not find container \"18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca\": container with ID starting with 18857021ba72785c65588b1dd7679365cc48a840d15877be6231a8357e04d9ca not found: ID does not exist"
Oct 30 21:32:08.688994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.688967    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"]
Oct 30 21:32:08.694269 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.694244    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-3a65ec-5cd567f4ff-b9kkk"]
Oct 30 21:32:08.702664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.702640    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"]
Oct 30 21:32:08.707827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.707803    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-3a65ec-66fc647b99-5kthb"]
Oct 30 21:32:08.815757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:08.815669    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:09.851859 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:09.851825    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:09.949370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:09.949340    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:10.004700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.004665    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd7151ac-50e1-411b-a52f-f65a6e246b94" path="/var/lib/kubelet/pods/bd7151ac-50e1-411b-a52f-f65a6e246b94/volumes"
Oct 30 21:32:10.005210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.005177    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6d83db6-f575-405f-b80b-4c65738bfb48" path="/var/lib/kubelet/pods/c6d83db6-f575-405f-b80b-4c65738bfb48/volumes"
Oct 30 21:32:10.189604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.189519    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:10.812442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.812405    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:10.858963 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.858884    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.892638    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-envoy\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.892724    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-socket\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.892917    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-data\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.892989    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b830a023-a9ed-4708-90d2-e2a946db163c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b830a023-a9ed-4708-90d2-e2a946db163c-istio-podinfo\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.893034    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-istio-token\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-istio-token\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.893093    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-kube-api-access-gpwd9\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-kube-api-access-gpwd9\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.893182    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b830a023-a9ed-4708-90d2-e2a946db163c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b830a023-a9ed-4708-90d2-e2a946db163c-istiod-ca-cert\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.893247    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-certs\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.893318    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-credential-socket\") pod \"b830a023-a9ed-4708-90d2-e2a946db163c\" (UID: \"b830a023-a9ed-4708-90d2-e2a946db163c\") "
Oct 30 21:32:10.895416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.894236    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-credential-socket" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:10.908452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.899203    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-envoy" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:10.908452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.899870    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-socket" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:10.908452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.900116    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-data" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:10.913937 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.912546    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-certs" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:10.913937 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.912958    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b830a023-a9ed-4708-90d2-e2a946db163c-istiod-ca-cert" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:10.916040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.915647    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-istio-token" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:10.916944 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.916913    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b830a023-a9ed-4708-90d2-e2a946db163c-istio-podinfo" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:10.917846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.917823    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-kube-api-access-gpwd9" pod "b830a023-a9ed-4708-90d2-e2a946db163c" (UID: "b830a023-a9ed-4708-90d2-e2a946db163c"). InnerVolumeSpecName "kube-api-access-gpwd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:10.994979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.994935    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b830a023-a9ed-4708-90d2-e2a946db163c-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.994979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.994971    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.994979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.994983    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpwd9\" (UniqueName: \"kubernetes.io/projected/b830a023-a9ed-4708-90d2-e2a946db163c-kube-api-access-gpwd9\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.995233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.994992    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b830a023-a9ed-4708-90d2-e2a946db163c-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.995233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.995003    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.995233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.995023    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.995233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.995032    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.995233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.995040    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:10.995233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:10.995048    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b830a023-a9ed-4708-90d2-e2a946db163c-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:11.508677 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.508636    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" podUID="7937d42d-4d76-4908-a189-b3e03aa65932" containerName="payload-processing" containerID="cri-o://bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d" gracePeriod=25
Oct 30 21:32:11.526817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.526779    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" podUID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" containerName="payload-pre-processing" containerID="cri-o://3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09" gracePeriod=25
Oct 30 21:32:11.689114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.689081    4852 generic.go:356] "Generic (PLEG): container finished" podID="b830a023-a9ed-4708-90d2-e2a946db163c" containerID="a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23" exitCode=0
Oct 30 21:32:11.689264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.689145    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"
Oct 30 21:32:11.689264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.689169    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" event={"ID":"b830a023-a9ed-4708-90d2-e2a946db163c","Type":"ContainerDied","Data":"a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23"}
Oct 30 21:32:11.689264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.689230    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz" event={"ID":"b830a023-a9ed-4708-90d2-e2a946db163c","Type":"ContainerDied","Data":"d1a8ee84e9b98e4204bc6658b28b56e81c84c385518655c7875ae4dd3da0d5d7"}
Oct 30 21:32:11.689264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.689263    4852 scope.go:122] "RemoveContainer" containerID="a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23"
Oct 30 21:32:11.704972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.704908    4852 scope.go:122] "RemoveContainer" containerID="a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23"
Oct 30 21:32:11.705342 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:11.705317    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23\": container with ID starting with a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23 not found: ID does not exist" containerID="a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23"
Oct 30 21:32:11.705412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.705351    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23"} err="failed to get container status \"a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23\": rpc error: code = NotFound desc = could not find container \"a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23\": container with ID starting with a81189faf4bf2307b2f0e338b678cd8bcdef5c51ae589fa9bed4a500c4dcbb23 not found: ID does not exist"
Oct 30 21:32:11.714253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.714182    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"]
Oct 30 21:32:11.718579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.718557    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-8fa15b-openshift-default-6cf456b647-8mvmz"]
Oct 30 21:32:11.872466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:11.872372    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:12.006774 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:12.006743    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" path="/var/lib/kubelet/pods/b830a023-a9ed-4708-90d2-e2a946db163c/volumes"
Oct 30 21:32:12.692559 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:12.692511    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" podUID="6a24bc29-1110-4746-9bca-5be86231f077" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:12.799263 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:12.797896    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:13.163291 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:13.163250    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"]
Oct 30 21:32:13.168425 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:13.168362    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"]
Oct 30 21:32:13.189503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:13.189456    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"]
Oct 30 21:32:13.255906 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:13.255863    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:13.730615 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:13.730571    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:14.036529 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.036499    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:32:14.133649 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.133606    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a24bc29-1110-4746-9bca-5be86231f077-kube-api-access-dbz2b\" (UniqueName: \"kubernetes.io/projected/6a24bc29-1110-4746-9bca-5be86231f077-kube-api-access-dbz2b\") pod \"6a24bc29-1110-4746-9bca-5be86231f077\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") "
Oct 30 21:32:14.133862 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.133760    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a24bc29-1110-4746-9bca-5be86231f077-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a24bc29-1110-4746-9bca-5be86231f077-plugins-config-volume\") pod \"6a24bc29-1110-4746-9bca-5be86231f077\" (UID: \"6a24bc29-1110-4746-9bca-5be86231f077\") "
Oct 30 21:32:14.134124 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.134102    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a24bc29-1110-4746-9bca-5be86231f077-plugins-config-volume" pod "6a24bc29-1110-4746-9bca-5be86231f077" (UID: "6a24bc29-1110-4746-9bca-5be86231f077"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:14.135800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.135777    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a24bc29-1110-4746-9bca-5be86231f077-kube-api-access-dbz2b" pod "6a24bc29-1110-4746-9bca-5be86231f077" (UID: "6a24bc29-1110-4746-9bca-5be86231f077"). InnerVolumeSpecName "kube-api-access-dbz2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:14.235757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.235716    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a24bc29-1110-4746-9bca-5be86231f077-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:14.235757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.235753    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbz2b\" (UniqueName: \"kubernetes.io/projected/6a24bc29-1110-4746-9bca-5be86231f077-kube-api-access-dbz2b\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:14.520041 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.520000    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:14.713380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.713345    4852 generic.go:356] "Generic (PLEG): container finished" podID="6a24bc29-1110-4746-9bca-5be86231f077" containerID="63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214" exitCode=0
Oct 30 21:32:14.713616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.713421    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"
Oct 30 21:32:14.713616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.713443    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" event={"ID":"6a24bc29-1110-4746-9bca-5be86231f077","Type":"ContainerDied","Data":"63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214"}
Oct 30 21:32:14.713616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.713505    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6" event={"ID":"6a24bc29-1110-4746-9bca-5be86231f077","Type":"ContainerDied","Data":"9566d48216d7e16dac786fb4c60b359f0beb9cf1a33f8a0ac6a009a5f78d3162"}
Oct 30 21:32:14.713616 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.713535    4852 scope.go:122] "RemoveContainer" containerID="63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214"
Oct 30 21:32:14.724505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.724485    4852 scope.go:122] "RemoveContainer" containerID="63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214"
Oct 30 21:32:14.724847 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:14.724816    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214\": container with ID starting with 63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214 not found: ID does not exist" containerID="63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214"
Oct 30 21:32:14.724984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.724854    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214"} err="failed to get container status \"63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214\": rpc error: code = NotFound desc = could not find container \"63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214\": container with ID starting with 63e82ab9108e004e12e1f2f7ced0b745cd04ff2052e5cfbfda23bc5513587214 not found: ID does not exist"
Oct 30 21:32:14.740582 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.740556    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"]
Oct 30 21:32:14.745180 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:14.745158    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8fa15b-b445f959-v6nb6"]
Oct 30 21:32:15.298843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:15.298803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:15.923777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:15.923733    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.005267 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.005231    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a24bc29-1110-4746-9bca-5be86231f077" path="/var/lib/kubelet/pods/6a24bc29-1110-4746-9bca-5be86231f077/volumes"
Oct 30 21:32:16.094422 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.093002    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.183732 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.183627    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.276355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.276323    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.398656 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.398616    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.508584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.508542    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.700242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.700206    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.801031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:16.800944    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:17.192863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:17.192779    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:18.048159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:18.048126    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:18.593762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:18.593730    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"]
Oct 30 21:32:18.609607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:18.609571    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"]
Oct 30 21:32:18.665728 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:18.665668    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:18.766352 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:18.766322    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:19.102062 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:19.102028    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:19.155451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:19.155179    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" podUID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:19.156432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:19.156398    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" podUID="7937d42d-4d76-4908-a189-b3e03aa65932" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:19.941694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:19.941650    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:20.637439 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:20.637406    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:20.868972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:20.868934    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"]
Oct 30 21:32:20.933207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:20.933119    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:21.035348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:21.035311    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:21.426834 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:21.426794    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:21.557677 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:21.557642    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:32:22.049126 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:22.049075    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:22.400053 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:22.399960    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:22.500045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:22.499998    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:22.939021 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:22.938960    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:23.331507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:23.331465    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:23.595133 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:23.594995    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" podUID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" containerName="payload-processing" containerID="cri-o://a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df" gracePeriod=25
Oct 30 21:32:23.610547 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:23.610507    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" podUID="d490b798-a939-4597-b893-212a76b2f589" containerName="payload-pre-processing" containerID="cri-o://ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68" gracePeriod=25
Oct 30 21:32:24.276860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:24.276825    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:24.464219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:24.464174    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:24.637091 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:24.636946    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:24.835962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:24.835928    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.195225 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:25.195192    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.380413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:25.380305    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.482569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:25.482444    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.580557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:25.580524    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.670551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:25.670517    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.869565 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:25.869514    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw" podUID="0f059960-bfce-4cda-bf8c-e4be7abf437f" containerName="payload-processing" containerID="cri-o://97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9" gracePeriod=25
Oct 30 21:32:26.007807 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.007759    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:26.137895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.137830    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:32:26.154155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.154117    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f059960-bfce-4cda-bf8c-e4be7abf437f-kube-api-access-zb55m\" (UniqueName: \"kubernetes.io/projected/0f059960-bfce-4cda-bf8c-e4be7abf437f-kube-api-access-zb55m\") pod \"0f059960-bfce-4cda-bf8c-e4be7abf437f\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") "
Oct 30 21:32:26.154353 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.154202    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f059960-bfce-4cda-bf8c-e4be7abf437f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f059960-bfce-4cda-bf8c-e4be7abf437f-plugins-config-volume\") pod \"0f059960-bfce-4cda-bf8c-e4be7abf437f\" (UID: \"0f059960-bfce-4cda-bf8c-e4be7abf437f\") "
Oct 30 21:32:26.155775 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.155481    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f059960-bfce-4cda-bf8c-e4be7abf437f-plugins-config-volume" pod "0f059960-bfce-4cda-bf8c-e4be7abf437f" (UID: "0f059960-bfce-4cda-bf8c-e4be7abf437f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:26.157099 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.157068    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f059960-bfce-4cda-bf8c-e4be7abf437f-kube-api-access-zb55m" pod "0f059960-bfce-4cda-bf8c-e4be7abf437f" (UID: "0f059960-bfce-4cda-bf8c-e4be7abf437f"). InnerVolumeSpecName "kube-api-access-zb55m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:26.257166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.257131    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb55m\" (UniqueName: \"kubernetes.io/projected/0f059960-bfce-4cda-bf8c-e4be7abf437f-kube-api-access-zb55m\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.257166 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.257162    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f059960-bfce-4cda-bf8c-e4be7abf437f-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.608478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.608435    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" podUID="d490b798-a939-4597-b893-212a76b2f589" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:26.608885 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.608488    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" podUID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:26.652135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.652099    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:26.804359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.804317    4852 generic.go:356] "Generic (PLEG): container finished" podID="0f059960-bfce-4cda-bf8c-e4be7abf437f" containerID="97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9" exitCode=0
Oct 30 21:32:26.804569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.804379    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"
Oct 30 21:32:26.804569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.804429    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw" event={"ID":"0f059960-bfce-4cda-bf8c-e4be7abf437f","Type":"ContainerDied","Data":"97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9"}
Oct 30 21:32:26.804569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.804510    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw" event={"ID":"0f059960-bfce-4cda-bf8c-e4be7abf437f","Type":"ContainerDied","Data":"182f94addfa728b6528a6bcb52947b6ead8115c1c7226670c63ae9bc400d4bac"}
Oct 30 21:32:26.804569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.804540    4852 scope.go:122] "RemoveContainer" containerID="97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9"
Oct 30 21:32:26.815669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.815646    4852 scope.go:122] "RemoveContainer" containerID="97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9"
Oct 30 21:32:26.815993 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:26.815973    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9\": container with ID starting with 97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9 not found: ID does not exist" containerID="97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9"
Oct 30 21:32:26.816049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.816004    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9"} err="failed to get container status \"97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9\": rpc error: code = NotFound desc = could not find container \"97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9\": container with ID starting with 97e8aa5f281361bedcbcbdeb890d15f3b38316a585cdef3486d9a01214872fe9 not found: ID does not exist"
Oct 30 21:32:26.829225 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.829197    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"]
Oct 30 21:32:26.837078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:26.837051    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f8e2c3-fcb9cf6cc-cgzmw"]
Oct 30 21:32:27.522467 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:27.522434    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:28.005415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.005305    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f059960-bfce-4cda-bf8c-e4be7abf437f" path="/var/lib/kubelet/pods/0f059960-bfce-4cda-bf8c-e4be7abf437f/volumes"
Oct 30 21:32:28.186081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.186021    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main" containerID="cri-o://ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f" gracePeriod=45
Oct 30 21:32:28.189698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.189663    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main" containerID="cri-o://15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4" gracePeriod=45
Oct 30 21:32:28.211082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.211041    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main" containerID="cri-o://a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1" gracePeriod=45
Oct 30 21:32:28.217947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.217916    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:28.486856 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.486826    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:32:28.503150 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.503122    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:32:28.546433 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.545797    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:32:28.593716 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593681    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/144803a1-e592-4af8-bd4d-c4e33ddf401a-tls-certs\" (UniqueName: \"kubernetes.io/secret/144803a1-e592-4af8-bd4d-c4e33ddf401a-tls-certs\") pod \"144803a1-e592-4af8-bd4d-c4e33ddf401a\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593771    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-home\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-home\") pod \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593800    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47ff9971-1706-43a5-9123-e51ce4028da5-tls-certs\" (UniqueName: \"kubernetes.io/secret/47ff9971-1706-43a5-9123-e51ce4028da5-tls-certs\") pod \"47ff9971-1706-43a5-9123-e51ce4028da5\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593817    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-dshm\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-dshm\") pod \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593844    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/870f82ea-35a6-4f67-a2ce-d1c2983fd410-kube-api-access-25m4h\" (UniqueName: \"kubernetes.io/projected/870f82ea-35a6-4f67-a2ce-d1c2983fd410-kube-api-access-25m4h\") pod \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593884    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-home\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-home\") pod \"144803a1-e592-4af8-bd4d-c4e33ddf401a\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593930    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-dshm\") pod \"144803a1-e592-4af8-bd4d-c4e33ddf401a\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.593967    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-tmp-dir\") pod \"47ff9971-1706-43a5-9123-e51ce4028da5\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") "
Oct 30 21:32:28.594049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594019    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-dshm\") pod \"47ff9971-1706-43a5-9123-e51ce4028da5\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594086    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/144803a1-e592-4af8-bd4d-c4e33ddf401a-kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/144803a1-e592-4af8-bd4d-c4e33ddf401a-kube-api-access-7g2f2\") pod \"144803a1-e592-4af8-bd4d-c4e33ddf401a\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594109    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tls-certs\" (UniqueName: \"kubernetes.io/secret/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tls-certs\") pod \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594135    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tmp-dir\") pod \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594177    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47ff9971-1706-43a5-9123-e51ce4028da5-kube-api-access-bfts6\" (UniqueName: \"kubernetes.io/projected/47ff9971-1706-43a5-9123-e51ce4028da5-kube-api-access-bfts6\") pod \"47ff9971-1706-43a5-9123-e51ce4028da5\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594225    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-tmp-dir\") pod \"144803a1-e592-4af8-bd4d-c4e33ddf401a\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594252    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-model-cache\") pod \"47ff9971-1706-43a5-9123-e51ce4028da5\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594301    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-model-cache\") pod \"144803a1-e592-4af8-bd4d-c4e33ddf401a\" (UID: \"144803a1-e592-4af8-bd4d-c4e33ddf401a\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594330    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-model-cache\") pod \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\" (UID: \"870f82ea-35a6-4f67-a2ce-d1c2983fd410\") "
Oct 30 21:32:28.594667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.594378    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-home\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-home\") pod \"47ff9971-1706-43a5-9123-e51ce4028da5\" (UID: \"47ff9971-1706-43a5-9123-e51ce4028da5\") "
Oct 30 21:32:28.595171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.595104    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-home" pod "870f82ea-35a6-4f67-a2ce-d1c2983fd410" (UID: "870f82ea-35a6-4f67-a2ce-d1c2983fd410"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.595334 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.595306    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-home" pod "47ff9971-1706-43a5-9123-e51ce4028da5" (UID: "47ff9971-1706-43a5-9123-e51ce4028da5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.595588 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.595565    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tmp-dir" pod "870f82ea-35a6-4f67-a2ce-d1c2983fd410" (UID: "870f82ea-35a6-4f67-a2ce-d1c2983fd410"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.596483 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.596459    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-tmp-dir" pod "47ff9971-1706-43a5-9123-e51ce4028da5" (UID: "47ff9971-1706-43a5-9123-e51ce4028da5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.598209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.597592    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-home" pod "144803a1-e592-4af8-bd4d-c4e33ddf401a" (UID: "144803a1-e592-4af8-bd4d-c4e33ddf401a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.598209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.598135    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-tmp-dir" pod "144803a1-e592-4af8-bd4d-c4e33ddf401a" (UID: "144803a1-e592-4af8-bd4d-c4e33ddf401a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.598209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.597474    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870f82ea-35a6-4f67-a2ce-d1c2983fd410-kube-api-access-25m4h" pod "870f82ea-35a6-4f67-a2ce-d1c2983fd410" (UID: "870f82ea-35a6-4f67-a2ce-d1c2983fd410"). InnerVolumeSpecName "kube-api-access-25m4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:28.598446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.598219    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47ff9971-1706-43a5-9123-e51ce4028da5-tls-certs" pod "47ff9971-1706-43a5-9123-e51ce4028da5" (UID: "47ff9971-1706-43a5-9123-e51ce4028da5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:28.598446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.598287    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/144803a1-e592-4af8-bd4d-c4e33ddf401a-tls-certs" pod "144803a1-e592-4af8-bd4d-c4e33ddf401a" (UID: "144803a1-e592-4af8-bd4d-c4e33ddf401a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:28.598647 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.598608    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-model-cache" pod "870f82ea-35a6-4f67-a2ce-d1c2983fd410" (UID: "870f82ea-35a6-4f67-a2ce-d1c2983fd410"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.598761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.596362    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-model-cache" pod "47ff9971-1706-43a5-9123-e51ce4028da5" (UID: "47ff9971-1706-43a5-9123-e51ce4028da5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.599114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.598813    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-model-cache" pod "144803a1-e592-4af8-bd4d-c4e33ddf401a" (UID: "144803a1-e592-4af8-bd4d-c4e33ddf401a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.599909 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.599878    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-dshm" pod "870f82ea-35a6-4f67-a2ce-d1c2983fd410" (UID: "870f82ea-35a6-4f67-a2ce-d1c2983fd410"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.601272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.601244    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tls-certs" pod "870f82ea-35a6-4f67-a2ce-d1c2983fd410" (UID: "870f82ea-35a6-4f67-a2ce-d1c2983fd410"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:28.603088 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.601589    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-dshm" pod "47ff9971-1706-43a5-9123-e51ce4028da5" (UID: "47ff9971-1706-43a5-9123-e51ce4028da5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.603764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.603724    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-dshm" pod "144803a1-e592-4af8-bd4d-c4e33ddf401a" (UID: "144803a1-e592-4af8-bd4d-c4e33ddf401a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:28.621059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.618118    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47ff9971-1706-43a5-9123-e51ce4028da5-kube-api-access-bfts6" pod "47ff9971-1706-43a5-9123-e51ce4028da5" (UID: "47ff9971-1706-43a5-9123-e51ce4028da5"). InnerVolumeSpecName "kube-api-access-bfts6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:28.621059 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.620549    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/144803a1-e592-4af8-bd4d-c4e33ddf401a-kube-api-access-7g2f2" pod "144803a1-e592-4af8-bd4d-c4e33ddf401a" (UID: "144803a1-e592-4af8-bd4d-c4e33ddf401a"). InnerVolumeSpecName "kube-api-access-7g2f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:28.696159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696115    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696154    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144803a1-e592-4af8-bd4d-c4e33ddf401a-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696163    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696172    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47ff9971-1706-43a5-9123-e51ce4028da5-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696180    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696219    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25m4h\" (UniqueName: \"kubernetes.io/projected/870f82ea-35a6-4f67-a2ce-d1c2983fd410-kube-api-access-25m4h\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696230    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696250    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696258    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696266    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696274    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g2f2\" (UniqueName: \"kubernetes.io/projected/144803a1-e592-4af8-bd4d-c4e33ddf401a-kube-api-access-7g2f2\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696282    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696297    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696307    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfts6\" (UniqueName: \"kubernetes.io/projected/47ff9971-1706-43a5-9123-e51ce4028da5-kube-api-access-bfts6\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696314    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696324    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47ff9971-1706-43a5-9123-e51ce4028da5-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696333    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144803a1-e592-4af8-bd4d-c4e33ddf401a-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.696428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.696340    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/870f82ea-35a6-4f67-a2ce-d1c2983fd410-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:28.821069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.820971    4852 generic.go:356] "Generic (PLEG): container finished" podID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerID="a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1" exitCode=0
Oct 30 21:32:28.821069 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.821024    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" event={"ID":"870f82ea-35a6-4f67-a2ce-d1c2983fd410","Type":"ContainerDied","Data":"a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1"}
Oct 30 21:32:28.821308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.821080    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr" event={"ID":"870f82ea-35a6-4f67-a2ce-d1c2983fd410","Type":"ContainerDied","Data":"94d5b2d4fb3f5d0a151865c697d59f95f050f7219a7f65fc93b4dba16e306d3f"}
Oct 30 21:32:28.821308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.821102    4852 scope.go:122] "RemoveContainer" containerID="a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1"
Oct 30 21:32:28.821308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.821037    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"
Oct 30 21:32:28.822836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.822817    4852 generic.go:356] "Generic (PLEG): container finished" podID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerID="ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f" exitCode=0
Oct 30 21:32:28.823052 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.822894    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" event={"ID":"144803a1-e592-4af8-bd4d-c4e33ddf401a","Type":"ContainerDied","Data":"ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f"}
Oct 30 21:32:28.823186 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.823078    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6" event={"ID":"144803a1-e592-4af8-bd4d-c4e33ddf401a","Type":"ContainerDied","Data":"784c38e1a16b0804ca1f1e3db1e4d67329735e0f61dac9f3c56db4f85eccf14e"}
Oct 30 21:32:28.823186 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.822945    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"
Oct 30 21:32:28.824795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.824770    4852 generic.go:356] "Generic (PLEG): container finished" podID="47ff9971-1706-43a5-9123-e51ce4028da5" containerID="15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4" exitCode=0
Oct 30 21:32:28.824921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.824805    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" event={"ID":"47ff9971-1706-43a5-9123-e51ce4028da5","Type":"ContainerDied","Data":"15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4"}
Oct 30 21:32:28.824921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.824852    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm" event={"ID":"47ff9971-1706-43a5-9123-e51ce4028da5","Type":"ContainerDied","Data":"772ffb154a62763e83b2ec8e6baba787c73cc5a84a4eca37bd0373465d6d077e"}
Oct 30 21:32:28.825045 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.824948    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"
Oct 30 21:32:28.834472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.834445    4852 scope.go:122] "RemoveContainer" containerID="a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1"
Oct 30 21:32:28.834811 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:28.834791    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1\": container with ID starting with a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1 not found: ID does not exist" containerID="a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1"
Oct 30 21:32:28.834904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.834820    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1"} err="failed to get container status \"a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1\": rpc error: code = NotFound desc = could not find container \"a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1\": container with ID starting with a23585be92e91aaeef616d07cc754b86645afdd38863c813823c258ba487e4b1 not found: ID does not exist"
Oct 30 21:32:28.834904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.834840    4852 scope.go:122] "RemoveContainer" containerID="ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f"
Oct 30 21:32:28.843967 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.843944    4852 scope.go:122] "RemoveContainer" containerID="ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f"
Oct 30 21:32:28.844244 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:28.844223    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f\": container with ID starting with ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f not found: ID does not exist" containerID="ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f"
Oct 30 21:32:28.844315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.844250    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f"} err="failed to get container status \"ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f\": rpc error: code = NotFound desc = could not find container \"ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f\": container with ID starting with ad802c6705c7c7847705b14f5cf401995b269bb256305e20528e28af6f0a9c0f not found: ID does not exist"
Oct 30 21:32:28.844315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.844267    4852 scope.go:122] "RemoveContainer" containerID="15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4"
Oct 30 21:32:28.848860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.848839    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"]
Oct 30 21:32:28.854855 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.854824    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/simulator-w4-53aa47-kserve-79f45d4fc4-hvxfr"]
Oct 30 21:32:28.855311 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.855197    4852 scope.go:122] "RemoveContainer" containerID="15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4"
Oct 30 21:32:28.855777 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:28.855711    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4\": container with ID starting with 15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4 not found: ID does not exist" containerID="15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4"
Oct 30 21:32:28.855777 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.855745    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4"} err="failed to get container status \"15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4\": rpc error: code = NotFound desc = could not find container \"15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4\": container with ID starting with 15e0e6ea1756b173a3335a3b562a3c8c527d39b8fa970d6a96c79658c9a8f1f4 not found: ID does not exist"
Oct 30 21:32:28.865159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.865131    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"]
Oct 30 21:32:28.869678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.869657    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-2-w4-53aa47-kserve-977b56845-4gdp6"]
Oct 30 21:32:28.879919 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.879893    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"]
Oct 30 21:32:28.885897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:28.885875    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-53aa47/distinct-w4-53aa47-kserve-5fb9b4cd6b-878nm"]
Oct 30 21:32:29.086778 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:29.086687    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:29.161241 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:29.161201    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" podUID="7937d42d-4d76-4908-a189-b3e03aa65932" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:29.164812 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:29.164764    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" podUID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:29.894002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:29.893967    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:30.004694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.004649    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" path="/var/lib/kubelet/pods/144803a1-e592-4af8-bd4d-c4e33ddf401a/volumes"
Oct 30 21:32:30.005219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.005196    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" path="/var/lib/kubelet/pods/47ff9971-1706-43a5-9123-e51ce4028da5/volumes"
Oct 30 21:32:30.005737 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.005715    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" path="/var/lib/kubelet/pods/870f82ea-35a6-4f67-a2ce-d1c2983fd410/volumes"
Oct 30 21:32:30.422923 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.422885    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"]
Oct 30 21:32:30.429256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.426641    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:30.522713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.522660    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:30.690363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.690270    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:30.826891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:30.826857    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:31.200431 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.200399    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:31.841102 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.841073    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:32:31.844609 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.844586    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:32:31.867772 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.867743    4852 generic.go:356] "Generic (PLEG): container finished" podID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" containerID="3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09" exitCode=0
Oct 30 21:32:31.867988 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.867965    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" event={"ID":"0162ea80-1b44-4504-a24d-26eefcbdfe5b","Type":"ContainerDied","Data":"3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09"}
Oct 30 21:32:31.868056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.868023    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg" event={"ID":"0162ea80-1b44-4504-a24d-26eefcbdfe5b","Type":"ContainerDied","Data":"51ed5d8449794bbc3fb97667e9741305dcdd6e51070ea199feba3986861977c0"}
Oct 30 21:32:31.868056 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.868049    4852 scope.go:122] "RemoveContainer" containerID="3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09"
Oct 30 21:32:31.868169 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.868124    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"
Oct 30 21:32:31.870022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.869992    4852 generic.go:356] "Generic (PLEG): container finished" podID="7937d42d-4d76-4908-a189-b3e03aa65932" containerID="bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d" exitCode=0
Oct 30 21:32:31.870161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.870081    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" event={"ID":"7937d42d-4d76-4908-a189-b3e03aa65932","Type":"ContainerDied","Data":"bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d"}
Oct 30 21:32:31.870161 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.870123    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh" event={"ID":"7937d42d-4d76-4908-a189-b3e03aa65932","Type":"ContainerDied","Data":"37d145b0096b3251620063f213e9b1a7d04bc90a034288bfab7f8b2e5e81d954"}
Oct 30 21:32:31.870273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.870198    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"
Oct 30 21:32:31.881521 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.881499    4852 scope.go:122] "RemoveContainer" containerID="3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09"
Oct 30 21:32:31.882100 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:31.882078    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09\": container with ID starting with 3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09 not found: ID does not exist" containerID="3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09"
Oct 30 21:32:31.882286 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.882249    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09"} err="failed to get container status \"3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09\": rpc error: code = NotFound desc = could not find container \"3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09\": container with ID starting with 3caec87310678fb0e0e01d5436250dc5cfa9f531a3cc9cae6b518ad5619f2d09 not found: ID does not exist"
Oct 30 21:32:31.882450 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.882299    4852 scope.go:122] "RemoveContainer" containerID="bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d"
Oct 30 21:32:31.893679 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.893654    4852 scope.go:122] "RemoveContainer" containerID="bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d"
Oct 30 21:32:31.894049 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:31.894022    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d\": container with ID starting with bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d not found: ID does not exist" containerID="bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d"
Oct 30 21:32:31.894131 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.894061    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d"} err="failed to get container status \"bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d\": rpc error: code = NotFound desc = could not find container \"bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d\": container with ID starting with bf7c41815070352c0d7afa5b9a3618d699157e2140dfa4a39ec84bfee768647d not found: ID does not exist"
Oct 30 21:32:31.938013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.937979    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0162ea80-1b44-4504-a24d-26eefcbdfe5b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0162ea80-1b44-4504-a24d-26eefcbdfe5b-plugins-config-volume\") pod \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") "
Oct 30 21:32:31.938203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.938089    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7937d42d-4d76-4908-a189-b3e03aa65932-kube-api-access-fglmr\" (UniqueName: \"kubernetes.io/projected/7937d42d-4d76-4908-a189-b3e03aa65932-kube-api-access-fglmr\") pod \"7937d42d-4d76-4908-a189-b3e03aa65932\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") "
Oct 30 21:32:31.938264 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.938233    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0162ea80-1b44-4504-a24d-26eefcbdfe5b-kube-api-access-j6zkq\" (UniqueName: \"kubernetes.io/projected/0162ea80-1b44-4504-a24d-26eefcbdfe5b-kube-api-access-j6zkq\") pod \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\" (UID: \"0162ea80-1b44-4504-a24d-26eefcbdfe5b\") "
Oct 30 21:32:31.938319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.938288    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7937d42d-4d76-4908-a189-b3e03aa65932-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7937d42d-4d76-4908-a189-b3e03aa65932-plugins-config-volume\") pod \"7937d42d-4d76-4908-a189-b3e03aa65932\" (UID: \"7937d42d-4d76-4908-a189-b3e03aa65932\") "
Oct 30 21:32:31.938380 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.938356    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0162ea80-1b44-4504-a24d-26eefcbdfe5b-plugins-config-volume" pod "0162ea80-1b44-4504-a24d-26eefcbdfe5b" (UID: "0162ea80-1b44-4504-a24d-26eefcbdfe5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:31.938746 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.938725    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0162ea80-1b44-4504-a24d-26eefcbdfe5b-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:31.938868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.938742    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7937d42d-4d76-4908-a189-b3e03aa65932-plugins-config-volume" pod "7937d42d-4d76-4908-a189-b3e03aa65932" (UID: "7937d42d-4d76-4908-a189-b3e03aa65932"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:31.940261 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.940239    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7937d42d-4d76-4908-a189-b3e03aa65932-kube-api-access-fglmr" pod "7937d42d-4d76-4908-a189-b3e03aa65932" (UID: "7937d42d-4d76-4908-a189-b3e03aa65932"). InnerVolumeSpecName "kube-api-access-fglmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:31.940364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:31.940285    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0162ea80-1b44-4504-a24d-26eefcbdfe5b-kube-api-access-j6zkq" pod "0162ea80-1b44-4504-a24d-26eefcbdfe5b" (UID: "0162ea80-1b44-4504-a24d-26eefcbdfe5b"). InnerVolumeSpecName "kube-api-access-j6zkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:32.027074 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.027045    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:32.040200 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.040169    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fglmr\" (UniqueName: \"kubernetes.io/projected/7937d42d-4d76-4908-a189-b3e03aa65932-kube-api-access-fglmr\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:32.040330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.040203    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6zkq\" (UniqueName: \"kubernetes.io/projected/0162ea80-1b44-4504-a24d-26eefcbdfe5b-kube-api-access-j6zkq\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:32.040330 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.040217    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7937d42d-4d76-4908-a189-b3e03aa65932-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:32.186579 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.186481    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"]
Oct 30 21:32:32.191993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.191961    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-a50675-7c46f9867f-q6vrg"]
Oct 30 21:32:32.203137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.203108    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"]
Oct 30 21:32:32.209185 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.209148    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-a50675-bc4d4d67f-lzfvh"]
Oct 30 21:32:32.914429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:32.914313    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:33.212700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:33.212605    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:33.313570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:33.313538    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:33.543682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:33.543645    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"]
Oct 30 21:32:33.543950 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:33.543923    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" containerID="cri-o://62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359" gracePeriod=30
Oct 30 21:32:33.571287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:33.571250    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:33.809804 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:33.809711    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:34.004336 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:34.004300    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" path="/var/lib/kubelet/pods/0162ea80-1b44-4504-a24d-26eefcbdfe5b/volumes"
Oct 30 21:32:34.004816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:34.004800    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7937d42d-4d76-4908-a189-b3e03aa65932" path="/var/lib/kubelet/pods/7937d42d-4d76-4908-a189-b3e03aa65932/volumes"
Oct 30 21:32:34.828626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:34.828553    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:35.563486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:35.563447    4852 patch_prober.go:29] interesting pod/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:32:35.563867 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:35.563512    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:32:35.728766 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:35.728708    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:35.825807 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:35.825718    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"]
Oct 30 21:32:35.891685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:35.891648    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:35.988426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:35.988337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.180199 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.180112    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"]
Oct 30 21:32:36.197322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.197287    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"]
Oct 30 21:32:36.284399 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.284345    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.611112 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.611071    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" podUID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:36.612327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.612284    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" podUID="d490b798-a939-4597-b893-212a76b2f589" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:36.662094 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.662060    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.760699 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.760658    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.860142 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:36.860110    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:37.403395 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:37.403345    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:37.535636 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:37.535604    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.324001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.323966    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.502412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.496859    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.594991 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:38.594959    4852 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd867c0a9_5a70_4024_8481_35f6f5c545db.slice/crio-62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 21:32:38.612255 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.612223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.721512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.721483    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.834238 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.834171    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:32:38.928288 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.928245    4852 generic.go:356] "Generic (PLEG): container finished" podID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerID="62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359" exitCode=0
Oct 30 21:32:38.928486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.928307    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" event={"ID":"d867c0a9-5a70-4024-8481-35f6f5c545db","Type":"ContainerDied","Data":"62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359"}
Oct 30 21:32:38.928486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.928350    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"
Oct 30 21:32:38.928486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.928362    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt" event={"ID":"d867c0a9-5a70-4024-8481-35f6f5c545db","Type":"ContainerDied","Data":"298ae3734ecacc68301433e8ce757dccd0a2b899bded8eb83af5fbca17db4355"}
Oct 30 21:32:38.928486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.928414    4852 scope.go:122] "RemoveContainer" containerID="62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359"
Oct 30 21:32:38.939128 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.939101    4852 scope.go:122] "RemoveContainer" containerID="62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359"
Oct 30 21:32:38.939504 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:38.939479    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359\": container with ID starting with 62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359 not found: ID does not exist" containerID="62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359"
Oct 30 21:32:38.939602 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.939512    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359"} err="failed to get container status \"62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359\": rpc error: code = NotFound desc = could not find container \"62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359\": container with ID starting with 62664a7a148bd9ef436812d6ea45c1859c68a82a1a56e0011335b43452594359 not found: ID does not exist"
Oct 30 21:32:38.963633 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963599    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-certs\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.963836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963665    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-podinfo\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.963836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963686    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-token\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-token\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.963836 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963741    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-envoy\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.963990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963847    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-credential-socket\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.963990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963906    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-kube-api-access-5cpdl\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-kube-api-access-5cpdl\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.963990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963919    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-certs" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:38.963990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.963974    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-socket\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.964519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964026    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-data\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.964519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964063    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d867c0a9-5a70-4024-8481-35f6f5c545db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d867c0a9-5a70-4024-8481-35f6f5c545db-istiod-ca-cert\") pod \"d867c0a9-5a70-4024-8481-35f6f5c545db\" (UID: \"d867c0a9-5a70-4024-8481-35f6f5c545db\") "
Oct 30 21:32:38.964519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964116    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-credential-socket" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:38.964519 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964205    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-socket" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:38.964759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964662    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d867c0a9-5a70-4024-8481-35f6f5c545db-istiod-ca-cert" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:38.964984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964910    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:38.964984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964937    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d867c0a9-5a70-4024-8481-35f6f5c545db-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:38.964984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964954    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:38.964984 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964449    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-data" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:38.965289 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.964967    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:38.966316 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.966292    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-podinfo" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:38.966569 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.966548    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-token" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:38.967485 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.967458    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-kube-api-access-5cpdl" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "kube-api-access-5cpdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:38.967589 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:38.967501    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-envoy" pod "d867c0a9-5a70-4024-8481-35f6f5c545db" (UID: "d867c0a9-5a70-4024-8481-35f6f5c545db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:39.066312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.066271    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:39.066312 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.066307    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:39.066562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.066320    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:39.066562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.066338    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cpdl\" (UniqueName: \"kubernetes.io/projected/d867c0a9-5a70-4024-8481-35f6f5c545db-kube-api-access-5cpdl\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:39.066562 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.066354    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d867c0a9-5a70-4024-8481-35f6f5c545db-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:39.163854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.163753    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:39.254617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.254579    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"]
Oct 30 21:32:39.257522 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.257496    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f8e2c3-openshift-default-9b5c4b479-nj9xt"]
Oct 30 21:32:39.621031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:39.620990    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:40.004363 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:40.004279    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" path="/var/lib/kubelet/pods/d867c0a9-5a70-4024-8481-35f6f5c545db/volumes"
Oct 30 21:32:40.504012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:40.503850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:40.826686 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:40.826634    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" podUID="c70bde44-f354-4a89-ac63-af5e6300f8fc" containerName="payload-processing" containerID="cri-o://97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3" gracePeriod=25
Oct 30 21:32:41.458093 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:41.458053    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.557587 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:41.557554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.657654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:41.657624    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:42.269743 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:42.269703    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:43.108704 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.108670    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:43.899089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.899067    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 21:32:43.901229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.901200    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:43.902654 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.902636    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 21:32:43.923335 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.923304    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d490b798-a939-4597-b893-212a76b2f589-kube-api-access-pgsvs\" (UniqueName: \"kubernetes.io/projected/d490b798-a939-4597-b893-212a76b2f589-kube-api-access-pgsvs\") pod \"d490b798-a939-4597-b893-212a76b2f589\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") "
Oct 30 21:32:43.923540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.923368    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-plugins-config-volume\") pod \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") "
Oct 30 21:32:43.923540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.923448    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d490b798-a939-4597-b893-212a76b2f589-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d490b798-a939-4597-b893-212a76b2f589-plugins-config-volume\") pod \"d490b798-a939-4597-b893-212a76b2f589\" (UID: \"d490b798-a939-4597-b893-212a76b2f589\") "
Oct 30 21:32:43.923540 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.923524    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-kube-api-access-hvqp4\" (UniqueName: \"kubernetes.io/projected/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-kube-api-access-hvqp4\") pod \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\" (UID: \"ac26d5f2-d10c-4a5b-ad45-44644b6f776a\") "
Oct 30 21:32:43.924078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.923833    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-plugins-config-volume" pod "ac26d5f2-d10c-4a5b-ad45-44644b6f776a" (UID: "ac26d5f2-d10c-4a5b-ad45-44644b6f776a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:43.924078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.923928    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d490b798-a939-4597-b893-212a76b2f589-plugins-config-volume" pod "d490b798-a939-4597-b893-212a76b2f589" (UID: "d490b798-a939-4597-b893-212a76b2f589"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:43.924078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.923988    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d490b798-a939-4597-b893-212a76b2f589-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:43.924078 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.924009    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:43.925962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.925922    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d490b798-a939-4597-b893-212a76b2f589-kube-api-access-pgsvs" pod "d490b798-a939-4597-b893-212a76b2f589" (UID: "d490b798-a939-4597-b893-212a76b2f589"). InnerVolumeSpecName "kube-api-access-pgsvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:43.926581 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.926560    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-kube-api-access-hvqp4" pod "ac26d5f2-d10c-4a5b-ad45-44644b6f776a" (UID: "ac26d5f2-d10c-4a5b-ad45-44644b6f776a"). InnerVolumeSpecName "kube-api-access-hvqp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:43.991471 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.991428    4852 generic.go:356] "Generic (PLEG): container finished" podID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" containerID="a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df" exitCode=0
Oct 30 21:32:43.991660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.991497    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" event={"ID":"ac26d5f2-d10c-4a5b-ad45-44644b6f776a","Type":"ContainerDied","Data":"a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df"}
Oct 30 21:32:43.991660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.991581    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6" event={"ID":"ac26d5f2-d10c-4a5b-ad45-44644b6f776a","Type":"ContainerDied","Data":"2465341eb2f014b799d1f0df5857b1b3bd91377edb2faec30d5f787a7185823d"}
Oct 30 21:32:43.991660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.991623    4852 scope.go:122] "RemoveContainer" containerID="a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df"
Oct 30 21:32:43.991875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.991824    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"
Oct 30 21:32:43.993823 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.993800    4852 generic.go:356] "Generic (PLEG): container finished" podID="d490b798-a939-4597-b893-212a76b2f589" containerID="ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68" exitCode=0
Oct 30 21:32:43.993954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.993930    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" event={"ID":"d490b798-a939-4597-b893-212a76b2f589","Type":"ContainerDied","Data":"ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68"}
Oct 30 21:32:43.994013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.993972    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p" event={"ID":"d490b798-a939-4597-b893-212a76b2f589","Type":"ContainerDied","Data":"ae7f9397429492f089e1a6bfd60730f6065a67c01acfa4e065416fe2eafe41cf"}
Oct 30 21:32:43.994013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:43.993974    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"
Oct 30 21:32:44.004207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.004188    4852 scope.go:122] "RemoveContainer" containerID="a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df"
Oct 30 21:32:44.004640 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:44.004478    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df\": container with ID starting with a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df not found: ID does not exist" containerID="a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df"
Oct 30 21:32:44.004640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.004520    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df"} err="failed to get container status \"a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df\": rpc error: code = NotFound desc = could not find container \"a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df\": container with ID starting with a8be8ed8408b87e88419da2ad5d7ed0b6101dc1505cb5dfde4ae863db748d2df not found: ID does not exist"
Oct 30 21:32:44.004640 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.004543    4852 scope.go:122] "RemoveContainer" containerID="ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68"
Oct 30 21:32:44.016894 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.016816    4852 scope.go:122] "RemoveContainer" containerID="ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68"
Oct 30 21:32:44.019346 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:44.019322    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68\": container with ID starting with ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68 not found: ID does not exist" containerID="ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68"
Oct 30 21:32:44.019518 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.019401    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68"} err="failed to get container status \"ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68\": rpc error: code = NotFound desc = could not find container \"ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68\": container with ID starting with ace1605e8b407c5066e309a70140c1e112d17decaa2fb580b275c62424fc4d68 not found: ID does not exist"
Oct 30 21:32:44.023163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.023141    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"]
Oct 30 21:32:44.025460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.025366    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgsvs\" (UniqueName: \"kubernetes.io/projected/d490b798-a939-4597-b893-212a76b2f589-kube-api-access-pgsvs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:44.025460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.025405    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvqp4\" (UniqueName: \"kubernetes.io/projected/ac26d5f2-d10c-4a5b-ad45-44644b6f776a-kube-api-access-hvqp4\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:44.027886 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.027862    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-53aa47-67f966d869-ldll6"]
Oct 30 21:32:44.040925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.040898    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"]
Oct 30 21:32:44.047759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.047737    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-53aa47-76d657f955-rk78p"]
Oct 30 21:32:44.168066 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.167977    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"]
Oct 30 21:32:44.182334 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.182298    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"]
Oct 30 21:32:44.229625 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.229589    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.331188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.331149    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.634624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.634597    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.873449 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.873290    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.973290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:44.973248    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:45.438090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.438037    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:45.449282 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.449242    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" containerID="cri-o://b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f" gracePeriod=45
Oct 30 21:32:45.603935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.603895    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.84:8000/ready\": dial tcp 10.133.0.84:8000: connect: connection refused"
Oct 30 21:32:45.697326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.697262    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.761326    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-dshm\") pod \"2c01a375-cdb0-48e7-8575-8054efd4ead6\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") "
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.761382    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c01a375-cdb0-48e7-8575-8054efd4ead6-kube-api-access-tmlrj\" (UniqueName: \"kubernetes.io/projected/2c01a375-cdb0-48e7-8575-8054efd4ead6-kube-api-access-tmlrj\") pod \"2c01a375-cdb0-48e7-8575-8054efd4ead6\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") "
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.761437    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-home\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-home\") pod \"2c01a375-cdb0-48e7-8575-8054efd4ead6\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") "
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.761454    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-tmp-dir\") pod \"2c01a375-cdb0-48e7-8575-8054efd4ead6\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") "
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.761506    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-model-cache\") pod \"2c01a375-cdb0-48e7-8575-8054efd4ead6\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") "
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.761665    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c01a375-cdb0-48e7-8575-8054efd4ead6-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c01a375-cdb0-48e7-8575-8054efd4ead6-tls-certs\") pod \"2c01a375-cdb0-48e7-8575-8054efd4ead6\" (UID: \"2c01a375-cdb0-48e7-8575-8054efd4ead6\") "
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.762639    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-model-cache" pod "2c01a375-cdb0-48e7-8575-8054efd4ead6" (UID: "2c01a375-cdb0-48e7-8575-8054efd4ead6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.762727    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-tmp-dir" pod "2c01a375-cdb0-48e7-8575-8054efd4ead6" (UID: "2c01a375-cdb0-48e7-8575-8054efd4ead6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.764557 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.762965    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-home" pod "2c01a375-cdb0-48e7-8575-8054efd4ead6" (UID: "2c01a375-cdb0-48e7-8575-8054efd4ead6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.765321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.764618    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-dshm" pod "2c01a375-cdb0-48e7-8575-8054efd4ead6" (UID: "2c01a375-cdb0-48e7-8575-8054efd4ead6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.773838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.765505    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c01a375-cdb0-48e7-8575-8054efd4ead6-tls-certs" pod "2c01a375-cdb0-48e7-8575-8054efd4ead6" (UID: "2c01a375-cdb0-48e7-8575-8054efd4ead6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:45.773838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.765624    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c01a375-cdb0-48e7-8575-8054efd4ead6-kube-api-access-tmlrj" pod "2c01a375-cdb0-48e7-8575-8054efd4ead6" (UID: "2c01a375-cdb0-48e7-8575-8054efd4ead6"). InnerVolumeSpecName "kube-api-access-tmlrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:45.862947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.862904    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c01a375-cdb0-48e7-8575-8054efd4ead6-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.862947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.862938    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.862947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.862951    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmlrj\" (UniqueName: \"kubernetes.io/projected/2c01a375-cdb0-48e7-8575-8054efd4ead6-kube-api-access-tmlrj\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.863213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.862965    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.863213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.862976    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.863213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:45.862987    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c01a375-cdb0-48e7-8575-8054efd4ead6-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:46.005350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.005265    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" path="/var/lib/kubelet/pods/ac26d5f2-d10c-4a5b-ad45-44644b6f776a/volumes"
Oct 30 21:32:46.005896 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.005874    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d490b798-a939-4597-b893-212a76b2f589" path="/var/lib/kubelet/pods/d490b798-a939-4597-b893-212a76b2f589/volumes"
Oct 30 21:32:46.011223 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.011197    4852 generic.go:356] "Generic (PLEG): container finished" podID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerID="b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f" exitCode=0
Oct 30 21:32:46.011434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.011248    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"
Oct 30 21:32:46.011434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.011287    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" event={"ID":"2c01a375-cdb0-48e7-8575-8054efd4ead6","Type":"ContainerDied","Data":"b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f"}
Oct 30 21:32:46.011434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.011332    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl" event={"ID":"2c01a375-cdb0-48e7-8575-8054efd4ead6","Type":"ContainerDied","Data":"2d13cb580600cf7421af777981def1881550464d36ce806cf0d8fd8abd05fb14"}
Oct 30 21:32:46.011434 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.011352    4852 scope.go:122] "RemoveContainer" containerID="b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f"
Oct 30 21:32:46.028278 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.024023    4852 scope.go:122] "RemoveContainer" containerID="b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f"
Oct 30 21:32:46.028278 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:46.028242    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f\": container with ID starting with b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f not found: ID does not exist" containerID="b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f"
Oct 30 21:32:46.028428 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.028282    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f"} err="failed to get container status \"b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f\": rpc error: code = NotFound desc = could not find container \"b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f\": container with ID starting with b6c51f5e03bae03e87af04ffb75467ab09349bda798e2e838d57f9ec3e00a67f not found: ID does not exist"
Oct 30 21:32:46.036843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.036812    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"]
Oct 30 21:32:46.041711 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.041685    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-bc102e/distinct-2-w0-bc102e-kserve-7d9dc7677-hg4fl"]
Oct 30 21:32:46.216266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.216223    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:46.678837 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.678797    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:46.776413 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.769860    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:46.990667 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.990570    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"]
Oct 30 21:32:46.990850 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:46.990825    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy" containerID="cri-o://da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22" gracePeriod=30
Oct 30 21:32:47.020338 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:47.020302    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:47.110789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:47.110754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:47.742041 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:47.741989    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" podUID="c70bde44-f354-4a89-ac63-af5e6300f8fc" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:47.810635 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:47.810599    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:48.005275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.005197    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" path="/var/lib/kubelet/pods/2c01a375-cdb0-48e7-8575-8054efd4ead6/volumes"
Oct 30 21:32:48.384265 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.384228    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"]
Oct 30 21:32:48.384864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.384483    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z" podUID="4b4af4c2-7566-4a42-aa50-5ca11beaa48c" containerName="maas-api" containerID="cri-o://8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6" gracePeriod=30
Oct 30 21:32:48.440366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.440322    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:48.586355 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.586319    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:48.696598 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.696570    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:32:48.698207 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.698176    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:48.791594 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.791566    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:48.844751 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.844717    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-maas-api-tls\") pod \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") "
Oct 30 21:32:48.845001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.844931    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-kube-api-access-jh8pq\" (UniqueName: \"kubernetes.io/projected/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-kube-api-access-jh8pq\") pod \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") "
Oct 30 21:32:48.845001 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.844981    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-metrics-certs\") pod \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") "
Oct 30 21:32:48.845139 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.845096    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-openshift-service-ca\") pod \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\" (UID: \"4b4af4c2-7566-4a42-aa50-5ca11beaa48c\") "
Oct 30 21:32:48.846100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.846063    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-openshift-service-ca" pod "4b4af4c2-7566-4a42-aa50-5ca11beaa48c" (UID: "4b4af4c2-7566-4a42-aa50-5ca11beaa48c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:48.848368 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.848338    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-kube-api-access-jh8pq" pod "4b4af4c2-7566-4a42-aa50-5ca11beaa48c" (UID: "4b4af4c2-7566-4a42-aa50-5ca11beaa48c"). InnerVolumeSpecName "kube-api-access-jh8pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:48.849111 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.849071    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-metrics-certs" pod "4b4af4c2-7566-4a42-aa50-5ca11beaa48c" (UID: "4b4af4c2-7566-4a42-aa50-5ca11beaa48c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:48.854893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.854855    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-maas-api-tls" pod "4b4af4c2-7566-4a42-aa50-5ca11beaa48c" (UID: "4b4af4c2-7566-4a42-aa50-5ca11beaa48c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:48.947606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.947525    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh8pq\" (UniqueName: \"kubernetes.io/projected/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-kube-api-access-jh8pq\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:48.947606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.947556    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:48.947606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.947567    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:48.947606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:48.947577    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b4af4c2-7566-4a42-aa50-5ca11beaa48c-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:49.035888 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.035851    4852 generic.go:356] "Generic (PLEG): container finished" podID="4b4af4c2-7566-4a42-aa50-5ca11beaa48c" containerID="8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6" exitCode=0
Oct 30 21:32:49.036081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.035901    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"
Oct 30 21:32:49.036081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.035931    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z" event={"ID":"4b4af4c2-7566-4a42-aa50-5ca11beaa48c","Type":"ContainerDied","Data":"8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6"}
Oct 30 21:32:49.036081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.035984    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z" event={"ID":"4b4af4c2-7566-4a42-aa50-5ca11beaa48c","Type":"ContainerDied","Data":"e7a0af569b75d0e0a664decd0bc487e2cb0ada576ab6f4e272d4234527ce9127"}
Oct 30 21:32:49.036081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.036005    4852 scope.go:122] "RemoveContainer" containerID="8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6"
Oct 30 21:32:49.046790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.046770    4852 scope.go:122] "RemoveContainer" containerID="8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6"
Oct 30 21:32:49.047127 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:49.047106    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6\": container with ID starting with 8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6 not found: ID does not exist" containerID="8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6"
Oct 30 21:32:49.047175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.047137    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6"} err="failed to get container status \"8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6\": rpc error: code = NotFound desc = could not find container \"8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6\": container with ID starting with 8cd5b9bf49270908bb8934c833ac9d6437d244a9539a11d531b9c034d2596aa6 not found: ID does not exist"
Oct 30 21:32:49.060037 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.059995    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"]
Oct 30 21:32:49.062900 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.062870    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6741b1-8588f58b87-v8h2z"]
Oct 30 21:32:49.168592 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.168541    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" podUID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" containerName="payload-processing" containerID="cri-o://c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7" gracePeriod=25
Oct 30 21:32:49.175596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.175565    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.183478 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.183441    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" podUID="ba15be70-b3ba-49d5-b8fd-d4333a112656" containerName="payload-pre-processing" containerID="cri-o://27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86" gracePeriod=25
Oct 30 21:32:49.348603 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.348558    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"]
Oct 30 21:32:49.348909 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.348878    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h" podUID="27d5e04b-1973-4553-8026-555abdc93f9d" containerName="maas-api" containerID="cri-o://72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56" gracePeriod=30
Oct 30 21:32:49.395197 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.395163    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.603981 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.573347    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.671292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.671260    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.676373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.676350    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 21:32:49.769177 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.769133    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-maas-api-tls\") pod \"27d5e04b-1973-4553-8026-555abdc93f9d\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") "
Oct 30 21:32:49.769414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.769195    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27d5e04b-1973-4553-8026-555abdc93f9d-kube-api-access-hwrbq\" (UniqueName: \"kubernetes.io/projected/27d5e04b-1973-4553-8026-555abdc93f9d-kube-api-access-hwrbq\") pod \"27d5e04b-1973-4553-8026-555abdc93f9d\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") "
Oct 30 21:32:49.769414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.769285    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27d5e04b-1973-4553-8026-555abdc93f9d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d5e04b-1973-4553-8026-555abdc93f9d-openshift-service-ca\") pod \"27d5e04b-1973-4553-8026-555abdc93f9d\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") "
Oct 30 21:32:49.769414 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.769346    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-metrics-certs\") pod \"27d5e04b-1973-4553-8026-555abdc93f9d\" (UID: \"27d5e04b-1973-4553-8026-555abdc93f9d\") "
Oct 30 21:32:49.769713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.769687    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27d5e04b-1973-4553-8026-555abdc93f9d-openshift-service-ca" pod "27d5e04b-1973-4553-8026-555abdc93f9d" (UID: "27d5e04b-1973-4553-8026-555abdc93f9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:49.782526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.771850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.782526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.777582    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d5e04b-1973-4553-8026-555abdc93f9d-kube-api-access-hwrbq" pod "27d5e04b-1973-4553-8026-555abdc93f9d" (UID: "27d5e04b-1973-4553-8026-555abdc93f9d"). InnerVolumeSpecName "kube-api-access-hwrbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:49.783838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.783759    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-maas-api-tls" pod "27d5e04b-1973-4553-8026-555abdc93f9d" (UID: "27d5e04b-1973-4553-8026-555abdc93f9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:49.784173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.784142    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-metrics-certs" pod "27d5e04b-1973-4553-8026-555abdc93f9d" (UID: "27d5e04b-1973-4553-8026-555abdc93f9d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:49.870321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.870236    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d5e04b-1973-4553-8026-555abdc93f9d-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:49.870321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.870274    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:49.870321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.870284    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d5e04b-1973-4553-8026-555abdc93f9d-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:49.870321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:49.870293    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwrbq\" (UniqueName: \"kubernetes.io/projected/27d5e04b-1973-4553-8026-555abdc93f9d-kube-api-access-hwrbq\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:50.005790 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.005755    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b4af4c2-7566-4a42-aa50-5ca11beaa48c" path="/var/lib/kubelet/pods/4b4af4c2-7566-4a42-aa50-5ca11beaa48c/volumes"
Oct 30 21:32:50.043713 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.043679    4852 generic.go:356] "Generic (PLEG): container finished" podID="27d5e04b-1973-4553-8026-555abdc93f9d" containerID="72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56" exitCode=0
Oct 30 21:32:50.043893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.043761    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"
Oct 30 21:32:50.043893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.043765    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h" event={"ID":"27d5e04b-1973-4553-8026-555abdc93f9d","Type":"ContainerDied","Data":"72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56"}
Oct 30 21:32:50.043893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.043832    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h" event={"ID":"27d5e04b-1973-4553-8026-555abdc93f9d","Type":"ContainerDied","Data":"6daaab83ec3afc849e49e0532df5ad529b41a4ad126a06493c02720b639aa5a2"}
Oct 30 21:32:50.043893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.043861    4852 scope.go:122] "RemoveContainer" containerID="72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56"
Oct 30 21:32:50.054050 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.054025    4852 scope.go:122] "RemoveContainer" containerID="72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56"
Oct 30 21:32:50.054421 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:50.054398    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56\": container with ID starting with 72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56 not found: ID does not exist" containerID="72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56"
Oct 30 21:32:50.054503 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.054431    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56"} err="failed to get container status \"72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56\": rpc error: code = NotFound desc = could not find container \"72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56\": container with ID starting with 72911b96041ebd606c069503c4c7ee027c8991ef3520355a1c4968ee4e9b6d56 not found: ID does not exist"
Oct 30 21:32:50.066163 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.066136    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"]
Oct 30 21:32:50.069084 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.069061    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-bc102e-6d9b46764-bx74h"]
Oct 30 21:32:50.139664 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.139577    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:50.529954 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:50.529919    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:51.202068 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.202026    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main" containerID="cri-o://931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0" gracePeriod=45
Oct 30 21:32:51.221871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.221832    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main" containerID="cri-o://99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554" gracePeriod=45
Oct 30 21:32:51.301584 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.301555    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:51.548620 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.548273    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:32:51.579701 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.576382    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:32:51.579890 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:51.579825    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:32:51.693419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693357    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-home\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-home\") pod \"84cd2330-9d8e-4241-940b-af257ca600bc\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") "
Oct 30 21:32:51.693583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693431    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/66dd404d-7667-4c38-bff7-3dc844d263b7-tls-certs\" (UniqueName: \"kubernetes.io/secret/66dd404d-7667-4c38-bff7-3dc844d263b7-tls-certs\") pod \"66dd404d-7667-4c38-bff7-3dc844d263b7\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") "
Oct 30 21:32:51.693583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693481    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-model-cache\") pod \"66dd404d-7667-4c38-bff7-3dc844d263b7\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") "
Oct 30 21:32:51.693583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693516    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-home\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-home\") pod \"66dd404d-7667-4c38-bff7-3dc844d263b7\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") "
Oct 30 21:32:51.693583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693572    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84cd2330-9d8e-4241-940b-af257ca600bc-kube-api-access-rwrnc\" (UniqueName: \"kubernetes.io/projected/84cd2330-9d8e-4241-940b-af257ca600bc-kube-api-access-rwrnc\") pod \"84cd2330-9d8e-4241-940b-af257ca600bc\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") "
Oct 30 21:32:51.693845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693632    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-tmp-dir\") pod \"66dd404d-7667-4c38-bff7-3dc844d263b7\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") "
Oct 30 21:32:51.693845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693671    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-model-cache\") pod \"84cd2330-9d8e-4241-940b-af257ca600bc\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") "
Oct 30 21:32:51.693845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693740    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-home" pod "84cd2330-9d8e-4241-940b-af257ca600bc" (UID: "84cd2330-9d8e-4241-940b-af257ca600bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.693845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693760    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-dshm\") pod \"84cd2330-9d8e-4241-940b-af257ca600bc\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") "
Oct 30 21:32:51.693845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693819    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66dd404d-7667-4c38-bff7-3dc844d263b7-kube-api-access-75zdv\" (UniqueName: \"kubernetes.io/projected/66dd404d-7667-4c38-bff7-3dc844d263b7-kube-api-access-75zdv\") pod \"66dd404d-7667-4c38-bff7-3dc844d263b7\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") "
Oct 30 21:32:51.693845 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693804    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-model-cache" pod "66dd404d-7667-4c38-bff7-3dc844d263b7" (UID: "66dd404d-7667-4c38-bff7-3dc844d263b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.694175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693869    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/84cd2330-9d8e-4241-940b-af257ca600bc-tls-certs\" (UniqueName: \"kubernetes.io/secret/84cd2330-9d8e-4241-940b-af257ca600bc-tls-certs\") pod \"84cd2330-9d8e-4241-940b-af257ca600bc\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") "
Oct 30 21:32:51.694175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693871    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-home" pod "66dd404d-7667-4c38-bff7-3dc844d263b7" (UID: "66dd404d-7667-4c38-bff7-3dc844d263b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.694175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693902    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-tmp-dir\") pod \"84cd2330-9d8e-4241-940b-af257ca600bc\" (UID: \"84cd2330-9d8e-4241-940b-af257ca600bc\") "
Oct 30 21:32:51.694175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693962    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-tmp-dir" pod "66dd404d-7667-4c38-bff7-3dc844d263b7" (UID: "66dd404d-7667-4c38-bff7-3dc844d263b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.694175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.693974    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-dshm\") pod \"66dd404d-7667-4c38-bff7-3dc844d263b7\" (UID: \"66dd404d-7667-4c38-bff7-3dc844d263b7\") "
Oct 30 21:32:51.694175 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694054    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-model-cache" pod "84cd2330-9d8e-4241-940b-af257ca600bc" (UID: "84cd2330-9d8e-4241-940b-af257ca600bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.694605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694300    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-tmp-dir" pod "84cd2330-9d8e-4241-940b-af257ca600bc" (UID: "84cd2330-9d8e-4241-940b-af257ca600bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.694605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694498    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.694605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694527    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.694605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694547    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.694605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694560    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.694605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694575    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.694605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.694588    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.696268 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.696240    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84cd2330-9d8e-4241-940b-af257ca600bc-tls-certs" pod "84cd2330-9d8e-4241-940b-af257ca600bc" (UID: "84cd2330-9d8e-4241-940b-af257ca600bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:51.696365 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.696330    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66dd404d-7667-4c38-bff7-3dc844d263b7-tls-certs" pod "66dd404d-7667-4c38-bff7-3dc844d263b7" (UID: "66dd404d-7667-4c38-bff7-3dc844d263b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:51.696504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.696483    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-dshm" pod "84cd2330-9d8e-4241-940b-af257ca600bc" (UID: "84cd2330-9d8e-4241-940b-af257ca600bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.696578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.696517    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66dd404d-7667-4c38-bff7-3dc844d263b7-kube-api-access-75zdv" pod "66dd404d-7667-4c38-bff7-3dc844d263b7" (UID: "66dd404d-7667-4c38-bff7-3dc844d263b7"). InnerVolumeSpecName "kube-api-access-75zdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:51.696631 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.696590    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cd2330-9d8e-4241-940b-af257ca600bc-kube-api-access-rwrnc" pod "84cd2330-9d8e-4241-940b-af257ca600bc" (UID: "84cd2330-9d8e-4241-940b-af257ca600bc"). InnerVolumeSpecName "kube-api-access-rwrnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:51.696974 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.696959    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-dshm" pod "66dd404d-7667-4c38-bff7-3dc844d263b7" (UID: "66dd404d-7667-4c38-bff7-3dc844d263b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:51.796049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.796015    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75zdv\" (UniqueName: \"kubernetes.io/projected/66dd404d-7667-4c38-bff7-3dc844d263b7-kube-api-access-75zdv\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.796049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.796042    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84cd2330-9d8e-4241-940b-af257ca600bc-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.796049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.796052    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66dd404d-7667-4c38-bff7-3dc844d263b7-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.796299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.796063    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66dd404d-7667-4c38-bff7-3dc844d263b7-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.796299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.796076    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwrnc\" (UniqueName: \"kubernetes.io/projected/84cd2330-9d8e-4241-940b-af257ca600bc-kube-api-access-rwrnc\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:51.796299 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:51.796085    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84cd2330-9d8e-4241-940b-af257ca600bc-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.007314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.007288    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27d5e04b-1973-4553-8026-555abdc93f9d" path="/var/lib/kubelet/pods/27d5e04b-1973-4553-8026-555abdc93f9d/volumes"
Oct 30 21:32:52.018543 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.018518    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:52.061958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.061866    4852 generic.go:356] "Generic (PLEG): container finished" podID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerID="99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554" exitCode=0
Oct 30 21:32:52.061958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.061921    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"
Oct 30 21:32:52.061958 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.061946    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" event={"ID":"66dd404d-7667-4c38-bff7-3dc844d263b7","Type":"ContainerDied","Data":"99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554"}
Oct 30 21:32:52.062254 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.061993    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft" event={"ID":"66dd404d-7667-4c38-bff7-3dc844d263b7","Type":"ContainerDied","Data":"a612dcaba79af395ab3d6d5a372574c2cfad2b04bc3871c2305e31021d06bb2c"}
Oct 30 21:32:52.062254 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.062011    4852 scope.go:122] "RemoveContainer" containerID="99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554"
Oct 30 21:32:52.063853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.063832    4852 generic.go:356] "Generic (PLEG): container finished" podID="84cd2330-9d8e-4241-940b-af257ca600bc" containerID="931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0" exitCode=0
Oct 30 21:32:52.063987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.063877    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"
Oct 30 21:32:52.063987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.063895    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" event={"ID":"84cd2330-9d8e-4241-940b-af257ca600bc","Type":"ContainerDied","Data":"931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0"}
Oct 30 21:32:52.063987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.063915    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv" event={"ID":"84cd2330-9d8e-4241-940b-af257ca600bc","Type":"ContainerDied","Data":"5e1dc41007c506fb4b262f063b0c1197daa31ec2892bb98a6560fa7f2a6dca58"}
Oct 30 21:32:52.107593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.107557    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"]
Oct 30 21:32:52.113668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.113632    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/simulator-w3-f9a819-kserve-74897965c6-kr6ft"]
Oct 30 21:32:52.115196 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.115174    4852 scope.go:122] "RemoveContainer" containerID="99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554"
Oct 30 21:32:52.115601 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:52.115578    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554\": container with ID starting with 99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554 not found: ID does not exist" containerID="99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554"
Oct 30 21:32:52.115701 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.115609    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554"} err="failed to get container status \"99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554\": rpc error: code = NotFound desc = could not find container \"99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554\": container with ID starting with 99ec2d2072ebf0c830db0331586a450d557ff526b3f2dba8094111f41d1ad554 not found: ID does not exist"
Oct 30 21:32:52.115701 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.115631    4852 scope.go:122] "RemoveContainer" containerID="931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0"
Oct 30 21:32:52.125678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.125651    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"]
Oct 30 21:32:52.127463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.127443    4852 scope.go:122] "RemoveContainer" containerID="931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0"
Oct 30 21:32:52.127783 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:52.127763    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0\": container with ID starting with 931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0 not found: ID does not exist" containerID="931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0"
Oct 30 21:32:52.127864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.127792    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0"} err="failed to get container status \"931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0\": rpc error: code = NotFound desc = could not find container \"931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0\": container with ID starting with 931be30af54dcd8fdfb427ebf7ef4faedf36d0b62ef0b8f1a6c81371b7da5ed0 not found: ID does not exist"
Oct 30 21:32:52.129082 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.129061    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-f9a819/distinct-w3-f9a819-kserve-6dc58f668-lvczv"]
Oct 30 21:32:52.270522 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.270493    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 21:32:52.402090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.401982    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-envoy\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402037    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-podinfo\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402145    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-token\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-token\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402222    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-data\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402253    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-socket\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402298    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-certs\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402326    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9fb1229-a875-470f-b0a1-342cf3778c47-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9fb1229-a875-470f-b0a1-342cf3778c47-istiod-ca-cert\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402368    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-credential-socket\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402513    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-kube-api-access-nxvvk\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-kube-api-access-nxvvk\") pod \"a9fb1229-a875-470f-b0a1-342cf3778c47\" (UID: \"a9fb1229-a875-470f-b0a1-342cf3778c47\") "
Oct 30 21:32:52.402669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402586    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-data" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:52.402669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402595    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-socket" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:52.402875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402750    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-certs" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:52.402875 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402812    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-credential-socket" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:52.402994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402944    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fb1229-a875-470f-b0a1-342cf3778c47-istiod-ca-cert" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:52.403054 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.402993    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.403054 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.403014    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.403054 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.403029    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.403054 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.403044    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.404399 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.404358    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-envoy" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:52.404816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.404789    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-podinfo" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:52.404816 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.404811    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-kube-api-access-nxvvk" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "kube-api-access-nxvvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:52.404975 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.404838    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-token" pod "a9fb1229-a875-470f-b0a1-342cf3778c47" (UID: "a9fb1229-a875-470f-b0a1-342cf3778c47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:52.504637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.504598    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.504637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.504633    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.504637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.504643    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9fb1229-a875-470f-b0a1-342cf3778c47-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.504891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.504652    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxvvk\" (UniqueName: \"kubernetes.io/projected/a9fb1229-a875-470f-b0a1-342cf3778c47-kube-api-access-nxvvk\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.504891 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.504663    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9fb1229-a875-470f-b0a1-342cf3778c47-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:52.826002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:52.825966    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:53.073067 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.073036    4852 generic.go:356] "Generic (PLEG): container finished" podID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerID="da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22" exitCode=0
Oct 30 21:32:53.073266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.073084    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"
Oct 30 21:32:53.073266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.073125    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" event={"ID":"a9fb1229-a875-470f-b0a1-342cf3778c47","Type":"ContainerDied","Data":"da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22"}
Oct 30 21:32:53.073266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.073181    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5" event={"ID":"a9fb1229-a875-470f-b0a1-342cf3778c47","Type":"ContainerDied","Data":"8887ceafac9ee66ebacff6284b407ee0620e873cf55b4d0f25b79ff7a6f5776c"}
Oct 30 21:32:53.073266 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.073212    4852 scope.go:122] "RemoveContainer" containerID="da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22"
Oct 30 21:32:53.084158 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.084140    4852 scope.go:122] "RemoveContainer" containerID="da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22"
Oct 30 21:32:53.084475 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:32:53.084453    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22\": container with ID starting with da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22 not found: ID does not exist" containerID="da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22"
Oct 30 21:32:53.084535 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.084483    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22"} err="failed to get container status \"da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22\": rpc error: code = NotFound desc = could not find container \"da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22\": container with ID starting with da114be4fd44cb88b36dbd18f2a7ec612abd6cd30ee6f007afd4ccf801b9af22 not found: ID does not exist"
Oct 30 21:32:53.099002 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.098969    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"]
Oct 30 21:32:53.102948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.102921    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-53aa47-openshift-default-9cbf98959-82jg5"]
Oct 30 21:32:53.517204 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:53.517172    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:54.003930 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:54.003857    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" path="/var/lib/kubelet/pods/66dd404d-7667-4c38-bff7-3dc844d263b7/volumes"
Oct 30 21:32:54.004295 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:54.004265    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" path="/var/lib/kubelet/pods/84cd2330-9d8e-4241-940b-af257ca600bc/volumes"
Oct 30 21:32:54.004688 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:54.004673    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" path="/var/lib/kubelet/pods/a9fb1229-a875-470f-b0a1-342cf3778c47/volumes"
Oct 30 21:32:54.244014 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:54.243980    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:54.907031 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:54.906995    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:54.915318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:54.915280    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" podUID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:55.664694 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:55.664651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:55.923668 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:55.923580    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" podUID="ba15be70-b3ba-49d5-b8fd-d4333a112656" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:56.513695 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:56.513630    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:57.414762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:57.414731    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:57.527356 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:57.527327    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:57.627608 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:57.627554    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:57.741421 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:57.741319    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" podUID="c70bde44-f354-4a89-ac63-af5e6300f8fc" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:32:57.872617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:57.872565    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:58.291541 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:58.291502    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:58.381087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:58.381053    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:58.474912 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:58.474856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:58.639472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:58.639352    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"]
Oct 30 21:32:58.639639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:58.639585    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy" containerID="cri-o://da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa" gracePeriod=30
Oct 30 21:32:58.671611 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:58.671573    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:59.274940 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:59.274902    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:59.969340 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:32:59.969304    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:00.662245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:00.662200    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:00.874441 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:00.874380    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:00.991630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:00.991597    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:01.087710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.087680    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:01.101630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.101599    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 21:33:01.156143 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.156110    4852 generic.go:356] "Generic (PLEG): container finished" podID="c70bde44-f354-4a89-ac63-af5e6300f8fc" containerID="97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3" exitCode=0
Oct 30 21:33:01.156301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.156160    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"
Oct 30 21:33:01.156301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.156198    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" event={"ID":"c70bde44-f354-4a89-ac63-af5e6300f8fc","Type":"ContainerDied","Data":"97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3"}
Oct 30 21:33:01.156301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.156264    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9" event={"ID":"c70bde44-f354-4a89-ac63-af5e6300f8fc","Type":"ContainerDied","Data":"fe4fb2bffb4b2dc844eefef70f08c68b762966eac6d85fc49d893485fa41960e"}
Oct 30 21:33:01.156301 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.156298    4852 scope.go:122] "RemoveContainer" containerID="97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3"
Oct 30 21:33:01.166103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.166078    4852 scope.go:122] "RemoveContainer" containerID="97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3"
Oct 30 21:33:01.166467 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:01.166439    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3\": container with ID starting with 97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3 not found: ID does not exist" containerID="97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3"
Oct 30 21:33:01.166576 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.166479    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3"} err="failed to get container status \"97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3\": rpc error: code = NotFound desc = could not find container \"97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3\": container with ID starting with 97e5a071554b33c59dd07685dada2f18c2a9a16cffa5804422937c3557c5fcd3 not found: ID does not exist"
Oct 30 21:33:01.213506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.213378    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c70bde44-f354-4a89-ac63-af5e6300f8fc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c70bde44-f354-4a89-ac63-af5e6300f8fc-plugins-config-volume\") pod \"c70bde44-f354-4a89-ac63-af5e6300f8fc\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") "
Oct 30 21:33:01.213506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.213497    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c70bde44-f354-4a89-ac63-af5e6300f8fc-kube-api-access-6g476\" (UniqueName: \"kubernetes.io/projected/c70bde44-f354-4a89-ac63-af5e6300f8fc-kube-api-access-6g476\") pod \"c70bde44-f354-4a89-ac63-af5e6300f8fc\" (UID: \"c70bde44-f354-4a89-ac63-af5e6300f8fc\") "
Oct 30 21:33:01.213847 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.213821    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70bde44-f354-4a89-ac63-af5e6300f8fc-plugins-config-volume" pod "c70bde44-f354-4a89-ac63-af5e6300f8fc" (UID: "c70bde44-f354-4a89-ac63-af5e6300f8fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:01.215637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.215606    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70bde44-f354-4a89-ac63-af5e6300f8fc-kube-api-access-6g476" pod "c70bde44-f354-4a89-ac63-af5e6300f8fc" (UID: "c70bde44-f354-4a89-ac63-af5e6300f8fc"). InnerVolumeSpecName "kube-api-access-6g476". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:01.315070 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.315037    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c70bde44-f354-4a89-ac63-af5e6300f8fc-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:01.315070 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.315067    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g476\" (UniqueName: \"kubernetes.io/projected/c70bde44-f354-4a89-ac63-af5e6300f8fc-kube-api-access-6g476\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:01.476183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.476081    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:01.511417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.508147    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"]
Oct 30 21:33:01.512293 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:01.512245    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-bc102e-555d6fdb7c-9jmn9"]
Oct 30 21:33:02.003666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:02.003633    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c70bde44-f354-4a89-ac63-af5e6300f8fc" path="/var/lib/kubelet/pods/c70bde44-f354-4a89-ac63-af5e6300f8fc/volumes"
Oct 30 21:33:02.011652 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:02.011600    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:02.829684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:02.829647    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:03.528887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:03.528850    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:03.900120 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:03.900092    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 21:33:04.045705 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.045664    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-envoy\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.045897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.045719    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-data\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.045897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.045760    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-socket\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.045897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.045783    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-podinfo\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.045897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.045833    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-certs\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.046106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.045910    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-token\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-token\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.046106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.045960    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-kube-api-access-44mx5\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-kube-api-access-44mx5\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.046106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046000    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istiod-ca-cert\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.046106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046023    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-data" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:04.046106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046046    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-credential-socket\") pod \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\" (UID: \"df04b00d-07bc-453d-a7fa-2a8622fb5a0c\") "
Oct 30 21:33:04.046432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046108    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-socket" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:04.046432 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046368    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istiod-ca-cert" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:04.046819 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046496    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-credential-socket" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:04.046819 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046620    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.047018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046996    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.047205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.047186    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.047326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.047208    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.047326 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.046863    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-certs" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:04.048245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.048223    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-podinfo" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:33:04.048452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.048436    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-kube-api-access-44mx5" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "kube-api-access-44mx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:04.048567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.048464    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-envoy" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:04.048776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.048756    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-token" pod "df04b00d-07bc-453d-a7fa-2a8622fb5a0c" (UID: "df04b00d-07bc-453d-a7fa-2a8622fb5a0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:04.147864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.147784    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.147864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.147814    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.147864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.147824    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.147864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.147834    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44mx5\" (UniqueName: \"kubernetes.io/projected/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-kube-api-access-44mx5\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.147864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.147846    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df04b00d-07bc-453d-a7fa-2a8622fb5a0c-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:04.180582 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.180542    4852 generic.go:356] "Generic (PLEG): container finished" podID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerID="da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa" exitCode=0
Oct 30 21:33:04.180780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.180586    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" event={"ID":"df04b00d-07bc-453d-a7fa-2a8622fb5a0c","Type":"ContainerDied","Data":"da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa"}
Oct 30 21:33:04.180780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.180621    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"
Oct 30 21:33:04.180780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.180655    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7" event={"ID":"df04b00d-07bc-453d-a7fa-2a8622fb5a0c","Type":"ContainerDied","Data":"b31c642c300cf6ecbf8dfb42710cc52ed9ab205232f47f28e09c48719922d64e"}
Oct 30 21:33:04.180780 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.180682    4852 scope.go:122] "RemoveContainer" containerID="da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa"
Oct 30 21:33:04.191750 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.191722    4852 scope.go:122] "RemoveContainer" containerID="da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa"
Oct 30 21:33:04.192047 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:04.192028    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa\": container with ID starting with da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa not found: ID does not exist" containerID="da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa"
Oct 30 21:33:04.192115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.192056    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa"} err="failed to get container status \"da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa\": rpc error: code = NotFound desc = could not find container \"da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa\": container with ID starting with da211f35889ee2c8f21527e7941834ef4f6c192fe0ce2c921877f9ff81b2befa not found: ID does not exist"
Oct 30 21:33:04.207511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.207481    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"]
Oct 30 21:33:04.212032 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.212004    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-bc102e-openshift-default-6bfc9f987f-688n7"]
Oct 30 21:33:04.226890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.226859    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:04.915258 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:04.915223    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" podUID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:33:05.053593 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:05.053552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:05.846522 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:05.846487    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:05.923446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:05.923411    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" podUID="ba15be70-b3ba-49d5-b8fd-d4333a112656" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:33:05.948057 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:05.948025    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:06.004408 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:06.004364    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" path="/var/lib/kubelet/pods/df04b00d-07bc-453d-a7fa-2a8622fb5a0c/volumes"
Oct 30 21:33:06.645757 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:06.645705    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:07.528862 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:07.528828    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:08.326463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:08.326418    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:08.429293 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:08.429221    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:08.564253 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:08.564207    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:08.666849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:08.666755    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:09.149873 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.149837    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:09.439348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.439318    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:33:09.467016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.466984    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:33:09.619866 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.619824    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-kube-api-access-t5gts\" (UniqueName: \"kubernetes.io/projected/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-kube-api-access-t5gts\") pod \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") "
Oct 30 21:33:09.620281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.620004    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-plugins-config-volume\") pod \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\" (UID: \"a50dbd7f-3da6-40fa-b0d6-0f18db33671e\") "
Oct 30 21:33:09.620281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.620041    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba15be70-b3ba-49d5-b8fd-d4333a112656-kube-api-access-272g4\" (UniqueName: \"kubernetes.io/projected/ba15be70-b3ba-49d5-b8fd-d4333a112656-kube-api-access-272g4\") pod \"ba15be70-b3ba-49d5-b8fd-d4333a112656\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") "
Oct 30 21:33:09.620281 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.620090    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba15be70-b3ba-49d5-b8fd-d4333a112656-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba15be70-b3ba-49d5-b8fd-d4333a112656-plugins-config-volume\") pod \"ba15be70-b3ba-49d5-b8fd-d4333a112656\" (UID: \"ba15be70-b3ba-49d5-b8fd-d4333a112656\") "
Oct 30 21:33:09.620578 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.620550    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-plugins-config-volume" pod "a50dbd7f-3da6-40fa-b0d6-0f18db33671e" (UID: "a50dbd7f-3da6-40fa-b0d6-0f18db33671e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:09.620639 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.620579    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba15be70-b3ba-49d5-b8fd-d4333a112656-plugins-config-volume" pod "ba15be70-b3ba-49d5-b8fd-d4333a112656" (UID: "ba15be70-b3ba-49d5-b8fd-d4333a112656"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:09.622073 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.622051    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-kube-api-access-t5gts" pod "a50dbd7f-3da6-40fa-b0d6-0f18db33671e" (UID: "a50dbd7f-3da6-40fa-b0d6-0f18db33671e"). InnerVolumeSpecName "kube-api-access-t5gts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:09.622181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.622163    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba15be70-b3ba-49d5-b8fd-d4333a112656-kube-api-access-272g4" pod "ba15be70-b3ba-49d5-b8fd-d4333a112656" (UID: "ba15be70-b3ba-49d5-b8fd-d4333a112656"). InnerVolumeSpecName "kube-api-access-272g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:09.721853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.721750    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:09.721853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.721783    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-272g4\" (UniqueName: \"kubernetes.io/projected/ba15be70-b3ba-49d5-b8fd-d4333a112656-kube-api-access-272g4\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:09.721853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.721792    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba15be70-b3ba-49d5-b8fd-d4333a112656-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:09.721853 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.721801    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5gts\" (UniqueName: \"kubernetes.io/projected/a50dbd7f-3da6-40fa-b0d6-0f18db33671e-kube-api-access-t5gts\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:09.729651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.729617    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"]
Oct 30 21:33:09.729883 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.729854    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b" podUID="d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" containerName="maas-api" containerID="cri-o://ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e" gracePeriod=30
Oct 30 21:33:09.772262 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.772222    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:09.975513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:09.971646    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:10.023555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.023526    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:33:10.067666 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.067632    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:10.125829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.125793    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-metrics-certs\") pod \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") "
Oct 30 21:33:10.126013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.125864    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-openshift-service-ca\") pod \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") "
Oct 30 21:33:10.126013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.125961    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-kube-api-access-ftvdh\" (UniqueName: \"kubernetes.io/projected/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-kube-api-access-ftvdh\") pod \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") "
Oct 30 21:33:10.126013 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.126001    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-maas-api-tls\") pod \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\" (UID: \"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47\") "
Oct 30 21:33:10.126285 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.126245    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-openshift-service-ca" pod "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" (UID: "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:10.126504 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.126484    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:10.128081 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.128057    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-maas-api-tls" pod "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" (UID: "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:10.128154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.128096    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-metrics-certs" pod "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" (UID: "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:10.128154 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.128121    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-kube-api-access-ftvdh" pod "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" (UID: "d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47"). InnerVolumeSpecName "kube-api-access-ftvdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:10.185626 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.184268    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:10.227495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.227406    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:10.227495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.227433    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftvdh\" (UniqueName: \"kubernetes.io/projected/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-kube-api-access-ftvdh\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:10.227495 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.227443    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:10.229762 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.229730    4852 generic.go:356] "Generic (PLEG): container finished" podID="d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" containerID="ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e" exitCode=0
Oct 30 21:33:10.229916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.229780    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"
Oct 30 21:33:10.229916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.229812    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b" event={"ID":"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47","Type":"ContainerDied","Data":"ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e"}
Oct 30 21:33:10.229916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.229872    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b" event={"ID":"d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47","Type":"ContainerDied","Data":"9be873583e9a61ff9c3932a14e0a32e2ea14e109f25387bacb256feeeb0a6b31"}
Oct 30 21:33:10.229916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.229896    4852 scope.go:122] "RemoveContainer" containerID="ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e"
Oct 30 21:33:10.231350 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.231327    4852 generic.go:356] "Generic (PLEG): container finished" podID="ba15be70-b3ba-49d5-b8fd-d4333a112656" containerID="27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86" exitCode=0
Oct 30 21:33:10.231486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.231365    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"
Oct 30 21:33:10.231486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.231419    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" event={"ID":"ba15be70-b3ba-49d5-b8fd-d4333a112656","Type":"ContainerDied","Data":"27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86"}
Oct 30 21:33:10.231486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.231463    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv" event={"ID":"ba15be70-b3ba-49d5-b8fd-d4333a112656","Type":"ContainerDied","Data":"37171da7222286ad004e395756f496d5c548e00e6aeac281db41e9eba3c2dbd3"}
Oct 30 21:33:10.233181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.233121    4852 generic.go:356] "Generic (PLEG): container finished" podID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" containerID="c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7" exitCode=0
Oct 30 21:33:10.233181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.233142    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" event={"ID":"a50dbd7f-3da6-40fa-b0d6-0f18db33671e","Type":"ContainerDied","Data":"c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7"}
Oct 30 21:33:10.233181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.233180    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"
Oct 30 21:33:10.233375 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.233181    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc" event={"ID":"a50dbd7f-3da6-40fa-b0d6-0f18db33671e","Type":"ContainerDied","Data":"df22b59aca75a31c87c6fb973a424855640dc6dd9845cc350c7621b203ed43d2"}
Oct 30 21:33:10.248685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.248658    4852 scope.go:122] "RemoveContainer" containerID="ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e"
Oct 30 21:33:10.249166 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:10.249140    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e\": container with ID starting with ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e not found: ID does not exist" containerID="ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e"
Oct 30 21:33:10.249256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.249183    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e"} err="failed to get container status \"ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e\": rpc error: code = NotFound desc = could not find container \"ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e\": container with ID starting with ce3a8a58199817600248d2f7c2b1fb149d20496200731e8ea42f007b1c4a8c7e not found: ID does not exist"
Oct 30 21:33:10.249256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.249208    4852 scope.go:122] "RemoveContainer" containerID="27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86"
Oct 30 21:33:10.260018 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.259990    4852 scope.go:122] "RemoveContainer" containerID="27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86"
Oct 30 21:33:10.260401 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:10.260350    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86\": container with ID starting with 27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86 not found: ID does not exist" containerID="27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86"
Oct 30 21:33:10.260526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.260412    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86"} err="failed to get container status \"27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86\": rpc error: code = NotFound desc = could not find container \"27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86\": container with ID starting with 27ccbb3f899268e22460ddc84084ec0e258cfdede9fd548b4fb1ea5b00e95f86 not found: ID does not exist"
Oct 30 21:33:10.260526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.260432    4852 scope.go:122] "RemoveContainer" containerID="c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7"
Oct 30 21:33:10.262287 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.262261    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"]
Oct 30 21:33:10.266904 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.266881    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6741b1-6c786b975c-hcpkc"]
Oct 30 21:33:10.270613 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.270598    4852 scope.go:122] "RemoveContainer" containerID="c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7"
Oct 30 21:33:10.270854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.270835    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:10.270943 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:10.270878    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7\": container with ID starting with c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7 not found: ID does not exist" containerID="c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7"
Oct 30 21:33:10.270943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.270909    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7"} err="failed to get container status \"c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7\": rpc error: code = NotFound desc = could not find container \"c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7\": container with ID starting with c50754dce0ed779a5c4d65bc311187448ae0bae39bf0b8afae80e83e27e5b0a7 not found: ID does not exist"
Oct 30 21:33:10.280947 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.280920    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"]
Oct 30 21:33:10.283401 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.283363    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6741b1-fdc8444dc-sfmlv"]
Oct 30 21:33:10.308153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.302685    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"]
Oct 30 21:33:10.308153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.306820    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-d95261-66b99c4d96-jpk9b"]
Oct 30 21:33:10.539959 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.539927    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"]
Oct 30 21:33:10.540201 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.540172    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy" containerID="cri-o://7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117" gracePeriod=30
Oct 30 21:33:10.558817 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.558785    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:10.659134 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:10.659078    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:11.043838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:11.043804    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:11.353907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:11.353806    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:11.864870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:11.864831    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:12.004590 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:12.004553    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" path="/var/lib/kubelet/pods/a50dbd7f-3da6-40fa-b0d6-0f18db33671e/volumes"
Oct 30 21:33:12.004979 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:12.004965    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ba15be70-b3ba-49d5-b8fd-d4333a112656" path="/var/lib/kubelet/pods/ba15be70-b3ba-49d5-b8fd-d4333a112656/volumes"
Oct 30 21:33:12.005329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:12.005317    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" path="/var/lib/kubelet/pods/d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47/volumes"
Oct 30 21:33:12.621918 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:12.621878    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:12.730722 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:12.730687    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:13.441637 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:13.441591    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:14.232022 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:14.231981    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.013693 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.013650    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.111484 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.111452    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.748415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.748349    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.797401 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.797368    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 21:33:15.877006 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.876915    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-kube-api-access-wx2d2\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-kube-api-access-wx2d2\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.877183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877013    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-credential-socket\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.877183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877070    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-istio-token\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-istio-token\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.877183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877114    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-envoy\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.877183 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877162    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b83b0578-9da4-47b6-b952-16cb884318ef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83b0578-9da4-47b6-b952-16cb884318ef-istiod-ca-cert\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.877448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877197    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-data\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.877448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877252    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-certs\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.877448 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877336    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-credential-socket" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:15.877675 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877630    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-certs" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:15.877761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877676    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:15.877761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877671    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-data" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:15.877761 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.877747    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b83b0578-9da4-47b6-b952-16cb884318ef-istiod-ca-cert" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:15.879521 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.879491    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-kube-api-access-wx2d2" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "kube-api-access-wx2d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:15.879669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.879515    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-envoy" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:15.879719 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.879672    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-istio-token" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:15.978840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.978802    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b83b0578-9da4-47b6-b952-16cb884318ef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83b0578-9da4-47b6-b952-16cb884318ef-istio-podinfo\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.978840 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.978847    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-socket\") pod \"b83b0578-9da4-47b6-b952-16cb884318ef\" (UID: \"b83b0578-9da4-47b6-b952-16cb884318ef\") "
Oct 30 21:33:15.979086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.979067    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx2d2\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-kube-api-access-wx2d2\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:15.979086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.979081    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83b0578-9da4-47b6-b952-16cb884318ef-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:15.979168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.979090    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:15.979168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.979099    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83b0578-9da4-47b6-b952-16cb884318ef-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:15.979168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.979108    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:15.979168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.979117    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:15.979168 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.979140    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-socket" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:15.980846 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:15.980825    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b83b0578-9da4-47b6-b952-16cb884318ef-istio-podinfo" pod "b83b0578-9da4-47b6-b952-16cb884318ef" (UID: "b83b0578-9da4-47b6-b952-16cb884318ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:33:16.086583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.086513    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83b0578-9da4-47b6-b952-16cb884318ef-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:16.086583 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.086549    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83b0578-9da4-47b6-b952-16cb884318ef-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:16.284800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.284765    4852 generic.go:356] "Generic (PLEG): container finished" podID="b83b0578-9da4-47b6-b952-16cb884318ef" containerID="7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117" exitCode=0
Oct 30 21:33:16.284995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.284823    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"
Oct 30 21:33:16.284995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.284838    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" event={"ID":"b83b0578-9da4-47b6-b952-16cb884318ef","Type":"ContainerDied","Data":"7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117"}
Oct 30 21:33:16.284995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.284880    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd" event={"ID":"b83b0578-9da4-47b6-b952-16cb884318ef","Type":"ContainerDied","Data":"6f55a9724150e47daeddebdd09b8b25594c0568fd9f3916e39418702faf5f3da"}
Oct 30 21:33:16.284995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.284901    4852 scope.go:122] "RemoveContainer" containerID="7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117"
Oct 30 21:33:16.294555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.294530    4852 scope.go:122] "RemoveContainer" containerID="7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117"
Oct 30 21:33:16.294856 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:16.294836    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117\": container with ID starting with 7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117 not found: ID does not exist" containerID="7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117"
Oct 30 21:33:16.294921 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.294864    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117"} err="failed to get container status \"7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117\": rpc error: code = NotFound desc = could not find container \"7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117\": container with ID starting with 7275cd57120fe3f83428a1ad865d60eba47cf6d76526493ae0a6ff26b516b117 not found: ID does not exist"
Oct 30 21:33:16.310633 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.310601    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"]
Oct 30 21:33:16.315659 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.315635    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-f9a819-openshift-default-78c5969f59-j9hzd"]
Oct 30 21:33:16.742277 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:16.742232    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:17.547076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:17.547029    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.005505 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:18.005409    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" path="/var/lib/kubelet/pods/b83b0578-9da4-47b6-b952-16cb884318ef/volumes"
Oct 30 21:33:18.010514 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:18.010483    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.109315 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:18.109279    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.247960 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:18.247908    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.365256 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:18.365177    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"]
Oct 30 21:33:18.365469 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:18.365437    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy" containerID="cri-o://fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9" gracePeriod=30
Oct 30 21:33:18.393442 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:18.393368    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:19.035472 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035439    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"]
Oct 30 21:33:19.035863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035815    4852 cpu_manager.go:414] "containerMap: removing container" podUID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" containerName="payload-processing"
Oct 30 21:33:19.035863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035825    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" containerName="payload-processing"
Oct 30 21:33:19.035863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035840    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" containerName="maas-api"
Oct 30 21:33:19.035863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035846    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" containerName="maas-api"
Oct 30 21:33:19.035863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035853    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy"
Oct 30 21:33:19.035863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035859    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy"
Oct 30 21:33:19.035863 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035867    4852 cpu_manager.go:414] "containerMap: removing container" podUID="bd7151ac-50e1-411b-a52f-f65a6e246b94" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035872    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd7151ac-50e1-411b-a52f-f65a6e246b94" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035879    4852 cpu_manager.go:414] "containerMap: removing container" podUID="311556e8-83a5-4fbe-a7ec-5efcb6d0432f" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035883    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="311556e8-83a5-4fbe-a7ec-5efcb6d0432f" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035891    4852 cpu_manager.go:414] "containerMap: removing container" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035896    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035904    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035908    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035915    4852 cpu_manager.go:414] "containerMap: removing container" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035920    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035926    4852 cpu_manager.go:414] "containerMap: removing container" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035931    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035937    4852 cpu_manager.go:414] "containerMap: removing container" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035942    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035948    4852 cpu_manager.go:414] "containerMap: removing container" podUID="7937d42d-4d76-4908-a189-b3e03aa65932" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035953    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7937d42d-4d76-4908-a189-b3e03aa65932" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035959    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b69450b8-9338-46e2-9c5a-094fad128ea4" containerName="maas-api"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035964    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b69450b8-9338-46e2-9c5a-094fad128ea4" containerName="maas-api"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035969    4852 cpu_manager.go:414] "containerMap: removing container" podUID="6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" containerName="revoke-keys"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035974    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" containerName="revoke-keys"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035981    4852 cpu_manager.go:414] "containerMap: removing container" podUID="7894ad8b-11d2-480f-92f1-8d18a3c2ba62" containerName="maas-api"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035987    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7894ad8b-11d2-480f-92f1-8d18a3c2ba62" containerName="maas-api"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035992    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.035997    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036004    4852 cpu_manager.go:414] "containerMap: removing container" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036008    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036015    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036019    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036026    4852 cpu_manager.go:414] "containerMap: removing container" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036030    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036035    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036039    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036046    4852 cpu_manager.go:414] "containerMap: removing container" podUID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036050    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" containerName="payload-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036057    4852 cpu_manager.go:414] "containerMap: removing container" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036062    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036069    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036073    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036080    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036084    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036090    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ba15be70-b3ba-49d5-b8fd-d4333a112656" containerName="payload-pre-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036094    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba15be70-b3ba-49d5-b8fd-d4333a112656" containerName="payload-pre-processing"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036100    4852 cpu_manager.go:414] "containerMap: removing container" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036104    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main"
Oct 30 21:33:19.036090 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036110    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036115    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036122    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036126    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036134    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0335594b-ba42-4f48-96d0-3196c63389ad" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036138    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0335594b-ba42-4f48-96d0-3196c63389ad" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036144    4852 cpu_manager.go:414] "containerMap: removing container" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036149    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036155    4852 cpu_manager.go:414] "containerMap: removing container" podUID="4b4af4c2-7566-4a42-aa50-5ca11beaa48c" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036160    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b4af4c2-7566-4a42-aa50-5ca11beaa48c" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036165    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036169    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036175    4852 cpu_manager.go:414] "containerMap: removing container" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036179    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036188    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036192    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036198    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036202    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036208    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036211    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036218    4852 cpu_manager.go:414] "containerMap: removing container" podUID="3a0d7c61-76d4-4830-9181-5875507551f3" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036224    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a0d7c61-76d4-4830-9181-5875507551f3" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036232    4852 cpu_manager.go:414] "containerMap: removing container" podUID="72e54db2-6a16-44e6-a3a0-6b388c371acd" containerName="revoke-keys"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036236    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72e54db2-6a16-44e6-a3a0-6b388c371acd" containerName="revoke-keys"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036242    4852 cpu_manager.go:414] "containerMap: removing container" podUID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036246    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036254    4852 cpu_manager.go:414] "containerMap: removing container" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036258    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036264    4852 cpu_manager.go:414] "containerMap: removing container" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036268    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036275    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036279    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036286    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c70bde44-f354-4a89-ac63-af5e6300f8fc" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036290    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c70bde44-f354-4a89-ac63-af5e6300f8fc" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036297    4852 cpu_manager.go:414] "containerMap: removing container" podUID="0f059960-bfce-4cda-bf8c-e4be7abf437f" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036302    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f059960-bfce-4cda-bf8c-e4be7abf437f" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036311    4852 cpu_manager.go:414] "containerMap: removing container" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036316    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036323    4852 cpu_manager.go:414] "containerMap: removing container" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036327    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036332    4852 cpu_manager.go:414] "containerMap: removing container" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036336    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036342    4852 cpu_manager.go:414] "containerMap: removing container" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036347    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036354    4852 cpu_manager.go:414] "containerMap: removing container" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036358    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036364    4852 cpu_manager.go:414] "containerMap: removing container" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036368    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036375    4852 cpu_manager.go:414] "containerMap: removing container" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036379    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036406    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036414    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036422    4852 cpu_manager.go:414] "containerMap: removing container" podUID="6a24bc29-1110-4746-9bca-5be86231f077" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036427    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a24bc29-1110-4746-9bca-5be86231f077" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036433    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c27704f4-9744-46b0-9f3f-eb655a517c21" containerName="revoke-keys"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036438    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c27704f4-9744-46b0-9f3f-eb655a517c21" containerName="revoke-keys"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036447    4852 cpu_manager.go:414] "containerMap: removing container" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036452    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036456    4852 cpu_manager.go:414] "containerMap: removing container" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036461    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036467    4852 cpu_manager.go:414] "containerMap: removing container" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036471    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036478    4852 cpu_manager.go:414] "containerMap: removing container" podUID="27d5e04b-1973-4553-8026-555abdc93f9d" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036482    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27d5e04b-1973-4553-8026-555abdc93f9d" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036491    4852 cpu_manager.go:414] "containerMap: removing container" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036498    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036504    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d490b798-a939-4597-b893-212a76b2f589" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036509    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d490b798-a939-4597-b893-212a76b2f589" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036515    4852 cpu_manager.go:414] "containerMap: removing container" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036520    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036529    4852 cpu_manager.go:414] "containerMap: removing container" podUID="c6d83db6-f575-405f-b80b-4c65738bfb48" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036533    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6d83db6-f575-405f-b80b-4c65738bfb48" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036617    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="7937d42d-4d76-4908-a189-b3e03aa65932" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036629    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b69450b8-9338-46e2-9c5a-094fad128ea4" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036637    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" containerName="revoke-keys"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036648    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="7894ad8b-11d2-480f-92f1-8d18a3c2ba62" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036654    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036660    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="66dd404d-7667-4c38-bff7-3dc844d263b7" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036707    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036713    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036719    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036726    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e636851-f7e8-46d2-bd43-f7202a0fcb32" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036731    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d9690d9-dfb2-47c7-805e-0a84c8a89b42" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036738    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036744    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036749    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba15be70-b3ba-49d5-b8fd-d4333a112656" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036757    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="144803a1-e592-4af8-bd4d-c4e33ddf401a" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036762    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b83b0578-9da4-47b6-b952-16cb884318ef" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036768    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0162ea80-1b44-4504-a24d-26eefcbdfe5b" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036774    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0335594b-ba42-4f48-96d0-3196c63389ad" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036780    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036786    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b4af4c2-7566-4a42-aa50-5ca11beaa48c" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036791    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b830a023-a9ed-4708-90d2-e2a946db163c" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036797    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036804    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ec5fb08-ff18-48b2-9611-eee1690dbb07" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036809    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036814    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036819    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a0d7c61-76d4-4830-9181-5875507551f3" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036825    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="72e54db2-6a16-44e6-a3a0-6b388c371acd" containerName="revoke-keys"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036831    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac26d5f2-d10c-4a5b-ad45-44644b6f776a" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036837    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="df04b00d-07bc-453d-a7fa-2a8622fb5a0c" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036843    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9fb1229-a875-470f-b0a1-342cf3778c47" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036848    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036853    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c70bde44-f354-4a89-ac63-af5e6300f8fc" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036859    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f059960-bfce-4cda-bf8c-e4be7abf437f" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036865    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="564948ce-6be6-44e8-97b5-04b7c1e1ac80" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036870    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036876    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036882    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="84cd2330-9d8e-4241-940b-af257ca600bc" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036888    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036894    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f57cfad-f8d4-4b39-a024-dfefeba9b431" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036899    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036905    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c01a375-cdb0-48e7-8575-8054efd4ead6" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036910    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d867c0a9-5a70-4024-8481-35f6f5c545db" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036914    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a24bc29-1110-4746-9bca-5be86231f077" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036919    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036924    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c27704f4-9744-46b0-9f3f-eb655a517c21" containerName="revoke-keys"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036929    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036935    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036940    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="eebf6d42-2bac-44f1-bbd5-f7bc24981893" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036946    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="27d5e04b-1973-4553-8026-555abdc93f9d" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036952    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036957    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d490b798-a939-4597-b893-212a76b2f589" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036962    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="870f82ea-35a6-4f67-a2ce-d1c2983fd410" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036968    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6d83db6-f575-405f-b80b-4c65738bfb48" containerName="payload-pre-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036974    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="a50dbd7f-3da6-40fa-b0d6-0f18db33671e" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036979    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2cd2ca5-85eb-456f-b1a9-c3b8fc44fb47" containerName="maas-api"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036985    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0685cd7c-3d01-4213-97e7-2215faee05ac" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036990    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd7151ac-50e1-411b-a52f-f65a6e246b94" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.036995    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="311556e8-83a5-4fbe-a7ec-5efcb6d0432f" containerName="payload-processing"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.037001    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="969d8145-ba12-4e33-837e-5f13d0d71291" containerName="istio-proxy"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.037007    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="47ff9971-1706-43a5-9123-e51ce4028da5" containerName="main"
Oct 30 21:33:19.037682 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.037012    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="57bcee80-0a44-4c57-bc04-19616abc74ff" containerName="cleanup"
Oct 30 21:33:19.041848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.041827    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.046348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.046320    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:33:19.046526 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.046479    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kjd66\""
Oct 30 21:33:19.048496 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.048470    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"]
Oct 30 21:33:19.124313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.124274    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.124313 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.124322    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.124610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.124448    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkrqf\" (UniqueName: \"kubernetes.io/projected/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-kube-api-access-jkrqf\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.193290 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.193260    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:19.225660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.225624    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.225879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.225668    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.225879 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.225707    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkrqf\" (UniqueName: \"kubernetes.io/projected/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-kube-api-access-jkrqf\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.226461 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.226433    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.228236 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.228214    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.233828 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.233805    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkrqf\" (UniqueName: \"kubernetes.io/projected/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-kube-api-access-jkrqf\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.358004 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.357850    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:19.635849 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.635752    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"]
Oct 30 21:33:19.643905 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:33:19.643869    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19bd7c20_fc68_4215_9b00_5ef1629d3a5c.slice/crio-0dd98d8954bcdf308332f5d96ded4fb3ae2e8b6b5213a418ae91ed8aaf3b6086 WatchSource:0}: Error finding container 0dd98d8954bcdf308332f5d96ded4fb3ae2e8b6b5213a418ae91ed8aaf3b6086: Status 404 returned error can't find the container with id 0dd98d8954bcdf308332f5d96ded4fb3ae2e8b6b5213a418ae91ed8aaf3b6086
Oct 30 21:33:19.939511 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:19.939419    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:20.312265 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:20.312228    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c" event={"ID":"19bd7c20-fc68-4215-9b00-5ef1629d3a5c","Type":"ContainerStarted","Data":"05c044a408f083cb1c7ff54f5553d1e3556e14f332c426ec25df5dbbfca88df5"}
Oct 30 21:33:20.312758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:20.312288    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c" event={"ID":"19bd7c20-fc68-4215-9b00-5ef1629d3a5c","Type":"ContainerStarted","Data":"0dd98d8954bcdf308332f5d96ded4fb3ae2e8b6b5213a418ae91ed8aaf3b6086"}
Oct 30 21:33:20.328633 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:20.328585    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c" podStartSLOduration=1.328570289 podStartE2EDuration="1.328570289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:20.327210915 +0000 UTC m=+3308.441945104" watchObservedRunningTime="2026-10-30 21:33:20.328570289 +0000 UTC m=+3308.443304525"
Oct 30 21:33:20.629244 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:20.629154    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:21.320011 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:21.319974    4852 generic.go:356] "Generic (PLEG): container finished" podID="19bd7c20-fc68-4215-9b00-5ef1629d3a5c" containerID="05c044a408f083cb1c7ff54f5553d1e3556e14f332c426ec25df5dbbfca88df5" exitCode=0
Oct 30 21:33:21.320412 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:21.320029    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c" event={"ID":"19bd7c20-fc68-4215-9b00-5ef1629d3a5c","Type":"ContainerDied","Data":"05c044a408f083cb1c7ff54f5553d1e3556e14f332c426ec25df5dbbfca88df5"}
Oct 30 21:33:21.422553 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:21.422516    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:21.558460 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:21.558419    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:33:21.599800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:21.599700    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.144016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.143967    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.464860 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.464832    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:22.561909 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.561868    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-kube-api-access-jkrqf\" (UniqueName: \"kubernetes.io/projected/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-kube-api-access-jkrqf\") pod \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") "
Oct 30 21:33:22.561909 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.561911    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-xks-ca\") pod \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") "
Oct 30 21:33:22.562127 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.561957    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-openshift-ca\") pod \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\" (UID: \"19bd7c20-fc68-4215-9b00-5ef1629d3a5c\") "
Oct 30 21:33:22.562443 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.562407    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-openshift-ca" pod "19bd7c20-fc68-4215-9b00-5ef1629d3a5c" (UID: "19bd7c20-fc68-4215-9b00-5ef1629d3a5c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:22.564080 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.564056    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-xks-ca" pod "19bd7c20-fc68-4215-9b00-5ef1629d3a5c" (UID: "19bd7c20-fc68-4215-9b00-5ef1629d3a5c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:33:22.564178 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.564156    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-kube-api-access-jkrqf" pod "19bd7c20-fc68-4215-9b00-5ef1629d3a5c" (UID: "19bd7c20-fc68-4215-9b00-5ef1629d3a5c"). InnerVolumeSpecName "kube-api-access-jkrqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:22.662903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.662860    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkrqf\" (UniqueName: \"kubernetes.io/projected/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-kube-api-access-jkrqf\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:22.662903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.662896    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-xks-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:22.662903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.662906    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/19bd7c20-fc68-4215-9b00-5ef1629d3a5c-maas-api-openshift-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:22.881533 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:22.881495    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:23.334102 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.334065    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c" event={"ID":"19bd7c20-fc68-4215-9b00-5ef1629d3a5c","Type":"ContainerDied","Data":"0dd98d8954bcdf308332f5d96ded4fb3ae2e8b6b5213a418ae91ed8aaf3b6086"}
Oct 30 21:33:23.334317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.334103    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"
Oct 30 21:33:23.334317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.334113    4852 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dd98d8954bcdf308332f5d96ded4fb3ae2e8b6b5213a418ae91ed8aaf3b6086"
Oct 30 21:33:23.616848 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.616817    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:23.621348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.621325    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:33:23.677366 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.677306    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-token\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-token\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.677607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.677427    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-data\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.677607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.677488    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60f52450-d4d3-4903-8be4-c97eac38e4f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60f52450-d4d3-4903-8be4-c97eac38e4f9-istiod-ca-cert\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.677607 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.677519    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-kube-api-access-mqc8q\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-kube-api-access-mqc8q\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.680289    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-socket\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.680398    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-envoy\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.680453    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-podinfo\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.680486    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-certs\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.680552    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-credential-socket\") pod \"60f52450-d4d3-4903-8be4-c97eac38e4f9\" (UID: \"60f52450-d4d3-4903-8be4-c97eac38e4f9\") "
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.680664    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-data" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.681000    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-socket" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.681011    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.681203    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-credential-socket" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:23.681502 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.681315    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60f52450-d4d3-4903-8be4-c97eac38e4f9-istiod-ca-cert" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:23.682075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.681889    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-certs" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:23.683067 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.683026    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-token" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:23.686784 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.686750    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-kube-api-access-mqc8q" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "kube-api-access-mqc8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:23.687159 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.687129    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-envoy" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:23.687492 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.687467    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-podinfo" pod "60f52450-d4d3-4903-8be4-c97eac38e4f9" (UID: "60f52450-d4d3-4903-8be4-c97eac38e4f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:33:23.782344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782306    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.782344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782338    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.782344 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782348    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60f52450-d4d3-4903-8be4-c97eac38e4f9-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.782622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782357    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqc8q\" (UniqueName: \"kubernetes.io/projected/60f52450-d4d3-4903-8be4-c97eac38e4f9-kube-api-access-mqc8q\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.782622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782367    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.782622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782375    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.782622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782414    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60f52450-d4d3-4903-8be4-c97eac38e4f9-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:23.782622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:23.782423    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60f52450-d4d3-4903-8be4-c97eac38e4f9-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:24.312700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.312569    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:24.341651 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.341613    4852 generic.go:356] "Generic (PLEG): container finished" podID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerID="fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9" exitCode=0
Oct 30 21:33:24.341864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.341681    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"
Oct 30 21:33:24.341864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.341696    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" event={"ID":"60f52450-d4d3-4903-8be4-c97eac38e4f9","Type":"ContainerDied","Data":"fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9"}
Oct 30 21:33:24.341864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.341792    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5" event={"ID":"60f52450-d4d3-4903-8be4-c97eac38e4f9","Type":"ContainerDied","Data":"37638c94bbf47435e5c0aa6ec231390158578f89410bc791595d65a86c86e13b"}
Oct 30 21:33:24.341864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.341823    4852 scope.go:122] "RemoveContainer" containerID="fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9"
Oct 30 21:33:24.352029 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.352001    4852 scope.go:122] "RemoveContainer" containerID="5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb"
Oct 30 21:33:24.362252 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.362230    4852 scope.go:122] "RemoveContainer" containerID="fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9"
Oct 30 21:33:24.362621 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:24.362571    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9\": container with ID starting with fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9 not found: ID does not exist" containerID="fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9"
Oct 30 21:33:24.362747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.362629    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9"} err="failed to get container status \"fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9\": rpc error: code = NotFound desc = could not find container \"fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9\": container with ID starting with fc502a3b13bd20daee416ab6257605e96b38b89d97ae4c56812143942bab01f9 not found: ID does not exist"
Oct 30 21:33:24.362747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.362661    4852 scope.go:122] "RemoveContainer" containerID="5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb"
Oct 30 21:33:24.362960 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:24.362941    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb\": container with ID starting with 5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb not found: ID does not exist" containerID="5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb"
Oct 30 21:33:24.363003 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.362966    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb"} err="failed to get container status \"5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb\": rpc error: code = NotFound desc = could not find container \"5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb\": container with ID starting with 5195eaa122bb7e472e544509d8cd5bd7beeb3669ad13c5d8f41573a904656ceb not found: ID does not exist"
Oct 30 21:33:24.363462 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.363443    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"]
Oct 30 21:33:24.382948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:24.382910    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-d95261-openshift-default-664cbbcc4f-9cmc5"]
Oct 30 21:33:25.007171 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:25.007132    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:25.713086 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:25.713050    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:26.003155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:26.003078    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" path="/var/lib/kubelet/pods/60f52450-d4d3-4903-8be4-c97eac38e4f9/volumes"
Oct 30 21:33:26.420156 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:26.420119    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.110314 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:27.110261    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.816181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:27.816133    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:28.520318 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:28.520287    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.239499 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:29.239467    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.339570 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:29.339535    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:30.019333 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:30.019300    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:30.719367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:30.719330    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:31.421046 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:31.421008    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.036871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:32.036844    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.200810 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:32.200771    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.357916 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:32.357802    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"]
Oct 30 21:33:32.360987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:32.360949    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-e53c37-cdt2c"]
Oct 30 21:33:32.373275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:32.373177    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.460684 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:32.460651    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.872182 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:32.872127    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.230346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:33.230267    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:34.003486 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:34.003449    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19bd7c20-fc68-4215-9b00-5ef1629d3a5c" path="/var/lib/kubelet/pods/19bd7c20-fc68-4215-9b00-5ef1629d3a5c/volumes"
Oct 30 21:33:34.028555 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:34.028521    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:34.129359 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:34.129324    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:34.730507 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:34.730472    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.460763 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:35.460717    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:36.349847 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:36.349809    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:37.153402 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:37.153361    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:37.832321 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:37.832276    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:38.549282 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:38.549237    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.401233 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:39.401185    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.586946 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:39.586909    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.687843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:39.687754    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.957177 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:39.957091    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:40.130189 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:40.130146    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:41.024567 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:41.024527    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:41.707956 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:41.707916    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:42.524839 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:42.524799    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:43.214451 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:43.214412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:44.012320 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:44.012274    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:44.717506 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:44.717466    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:45.419308 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:45.419269    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:46.115512 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:46.115477    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:46.834900 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:46.834864    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:47.608838 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:47.608803    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:48.317708 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:48.317673    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:49.016490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:49.016450    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:49.709274 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:49.709239    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.418371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:50.418334    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.625575 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:50.625526    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.707643 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:50.707552    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.212606 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:51.212562    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.557317 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:33:51.557279    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:33:51.930347 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:51.930264    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.087023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:52.086985    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.175907 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:52.175871    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.250276 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:52.250191    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.673473 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:52.673428    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:53.221622 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:53.221581    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:53.930446 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:53.930412    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:54.744371 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:54.744337    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:55.522895 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:55.522852    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.323040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:56.323005    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.124756 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:57.124718    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.917710 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:57.917671    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:58.535861 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:58.535825    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:58.635995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:58.635963    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.337076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:59.337042    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.440012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:59.439981    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.537728 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:59.537689    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.657242 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:59.657144    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"]
Oct 30 21:33:59.657531 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:59.657489    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy" containerID="cri-o://bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d" gracePeriod=30
Oct 30 21:33:59.695172 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:33:59.695138    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:00.160145 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:00.160117    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:00.260075 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:00.260041    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.232489 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:01.232458    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.932191 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:01.932150    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:02.809854 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:02.809818    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:03.456407 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:03.456362    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:04.212871 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:04.212832    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:04.824660 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:04.824623    4852 patch_prober.go:29] interesting pod/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:34:04.824855 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:04.824707    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 30 21:34:04.918019 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:04.917990    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:34:04.927229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:04.927158    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:05.017605 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017569    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-envoy\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.017789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017614    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-data\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.017789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017645    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-podinfo\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.017789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017686    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71a8ec2e-6164-49dd-98d4-a23312d456f2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a8ec2e-6164-49dd-98d4-a23312d456f2-istiod-ca-cert\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.017789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017710    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-socket\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.018020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017876    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-data" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:05.018020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017885    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-kube-api-access-j4gjc\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-kube-api-access-j4gjc\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.018020 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.017942    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-certs\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.018190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018022    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-token\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-token\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.018190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018023    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-socket" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:05.018190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018094    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-credential-socket\") pod \"71a8ec2e-6164-49dd-98d4-a23312d456f2\" (UID: \"71a8ec2e-6164-49dd-98d4-a23312d456f2\") "
Oct 30 21:34:05.018190 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018143    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71a8ec2e-6164-49dd-98d4-a23312d456f2-istiod-ca-cert" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:05.018415 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018247    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-certs" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:05.018497 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018467    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-credential-socket" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:05.018689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018664    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.018776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018695    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.018776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018710    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.018776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018724    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a8ec2e-6164-49dd-98d4-a23312d456f2-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.018776 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.018739    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.019992 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.019970    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-envoy" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:05.020115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.020044    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-podinfo" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:34:05.020115 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.020073    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-kube-api-access-j4gjc" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "kube-api-access-j4gjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:05.020224 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.020207    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-token" pod "71a8ec2e-6164-49dd-98d4-a23312d456f2" (UID: "71a8ec2e-6164-49dd-98d4-a23312d456f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:05.119669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.119627    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4gjc\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-kube-api-access-j4gjc\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.119669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.119662    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.119669 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.119672    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.119925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.119681    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a8ec2e-6164-49dd-98d4-a23312d456f2-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:05.131121 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.131081    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"]
Oct 30 21:34:05.214452 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.214356    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:05.580509 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.580474    4852 generic.go:356] "Generic (PLEG): container finished" podID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerID="bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d" exitCode=0
Oct 30 21:34:05.580698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.580522    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"
Oct 30 21:34:05.580698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.580531    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" event={"ID":"71a8ec2e-6164-49dd-98d4-a23312d456f2","Type":"ContainerDied","Data":"bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d"}
Oct 30 21:34:05.580698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.580576    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x" event={"ID":"71a8ec2e-6164-49dd-98d4-a23312d456f2","Type":"ContainerDied","Data":"0a9b0fb28db3fb6a5903b38b28fc1a1077a09ab3b43a3a7300d25f90497c42dc"}
Oct 30 21:34:05.580698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.580597    4852 scope.go:122] "RemoveContainer" containerID="bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d"
Oct 30 21:34:05.590571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.590534    4852 scope.go:122] "RemoveContainer" containerID="bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d"
Oct 30 21:34:05.590854 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:34:05.590834    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d\": container with ID starting with bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d not found: ID does not exist" containerID="bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d"
Oct 30 21:34:05.590925 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.590863    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d"} err="failed to get container status \"bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d\": rpc error: code = NotFound desc = could not find container \"bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d\": container with ID starting with bae29de326f1f9e36e3f70bb770f911118170aec34d4c7680fc92645bc0d0f0d not found: ID does not exist"
Oct 30 21:34:05.603764 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.603735    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"]
Oct 30 21:34:05.607825 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.607799    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-321ccd-openshift-default-787dff9cd7-v4s5x"]
Oct 30 21:34:05.712272 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.712240    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:05.931577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:05.931474    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:06.002700 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:06.002665    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" path="/var/lib/kubelet/pods/71a8ec2e-6164-49dd-98d4-a23312d456f2/volumes"
Oct 30 21:34:06.420759 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:06.420705    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:07.122932 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:07.122863    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:07.820455 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:07.820336    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.617843 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:08.617810    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:09.329431 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:09.329347    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:10.029901 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:10.029858    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:10.433612 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:10.433524    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"]
Oct 30 21:34:10.464243 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:10.464203    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"]
Oct 30 21:34:10.508456 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:10.508420    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:10.608317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:10.608284    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:10.846364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:10.846334    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.434595 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:15.434471    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" podUID="979b5e79-af17-48e2-969a-96a6ef1b4a37" containerName="payload-processing" containerID="cri-o://7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964" gracePeriod=25
Oct 30 21:34:15.465372 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:15.465329    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerName="payload-pre-processing" containerID="cri-o://25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a" gracePeriod=25
Oct 30 21:34:15.718665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:15.718575    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:15.718665 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:15.718595    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" podUID="979b5e79-af17-48e2-969a-96a6ef1b4a37" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:20.151830 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.151764    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main" containerID="cri-o://4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd" gracePeriod=45
Oct 30 21:34:20.397327 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.397302    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:34:20.470993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.470919    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/775b7bb3-76c8-41fd-876b-f3646c25b77b-kube-api-access-h4zxs\" (UniqueName: \"kubernetes.io/projected/775b7bb3-76c8-41fd-876b-f3646c25b77b-kube-api-access-h4zxs\") pod \"775b7bb3-76c8-41fd-876b-f3646c25b77b\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") "
Oct 30 21:34:20.470993 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.470976    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-dshm\") pod \"775b7bb3-76c8-41fd-876b-f3646c25b77b\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") "
Oct 30 21:34:20.471229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471038    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-tmp-dir\") pod \"775b7bb3-76c8-41fd-876b-f3646c25b77b\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") "
Oct 30 21:34:20.471229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471080    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/775b7bb3-76c8-41fd-876b-f3646c25b77b-tls-certs\" (UniqueName: \"kubernetes.io/secret/775b7bb3-76c8-41fd-876b-f3646c25b77b-tls-certs\") pod \"775b7bb3-76c8-41fd-876b-f3646c25b77b\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") "
Oct 30 21:34:20.471229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471135    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-model-cache\") pod \"775b7bb3-76c8-41fd-876b-f3646c25b77b\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") "
Oct 30 21:34:20.471229 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471192    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-home\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-home\") pod \"775b7bb3-76c8-41fd-876b-f3646c25b77b\" (UID: \"775b7bb3-76c8-41fd-876b-f3646c25b77b\") "
Oct 30 21:34:20.471513 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471381    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-tmp-dir" pod "775b7bb3-76c8-41fd-876b-f3646c25b77b" (UID: "775b7bb3-76c8-41fd-876b-f3646c25b77b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:20.471580 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471511    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-model-cache" pod "775b7bb3-76c8-41fd-876b-f3646c25b77b" (UID: "775b7bb3-76c8-41fd-876b-f3646c25b77b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:20.471648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471604    4852 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-tmp-dir\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:20.471648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471626    4852 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-model-cache\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:20.471648 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.471624    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-home" pod "775b7bb3-76c8-41fd-876b-f3646c25b77b" (UID: "775b7bb3-76c8-41fd-876b-f3646c25b77b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:20.473152 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.473132    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-dshm" pod "775b7bb3-76c8-41fd-876b-f3646c25b77b" (UID: "775b7bb3-76c8-41fd-876b-f3646c25b77b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:20.473532 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.473509    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775b7bb3-76c8-41fd-876b-f3646c25b77b-tls-certs" pod "775b7bb3-76c8-41fd-876b-f3646c25b77b" (UID: "775b7bb3-76c8-41fd-876b-f3646c25b77b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:20.473617 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.473517    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775b7bb3-76c8-41fd-876b-f3646c25b77b-kube-api-access-h4zxs" pod "775b7bb3-76c8-41fd-876b-f3646c25b77b" (UID: "775b7bb3-76c8-41fd-876b-f3646c25b77b"). InnerVolumeSpecName "kube-api-access-h4zxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:20.572382 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.572349    4852 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/775b7bb3-76c8-41fd-876b-f3646c25b77b-tls-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:20.572382 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.572380    4852 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-home\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:20.572601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.572409    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4zxs\" (UniqueName: \"kubernetes.io/projected/775b7bb3-76c8-41fd-876b-f3646c25b77b-kube-api-access-h4zxs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:20.572601 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.572418    4852 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/775b7bb3-76c8-41fd-876b-f3646c25b77b-dshm\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:20.670657 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.670618    4852 generic.go:356] "Generic (PLEG): container finished" podID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerID="4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd" exitCode=0
Oct 30 21:34:20.670864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.670677    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"
Oct 30 21:34:20.670864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.670707    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" event={"ID":"775b7bb3-76c8-41fd-876b-f3646c25b77b","Type":"ContainerDied","Data":"4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd"}
Oct 30 21:34:20.670864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.670751    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4" event={"ID":"775b7bb3-76c8-41fd-876b-f3646c25b77b","Type":"ContainerDied","Data":"fb6f9d09f150bd534c3b74077fd32fd5ad2caa7e090e1dba57c9771d94173f91"}
Oct 30 21:34:20.670864 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.670776    4852 scope.go:122] "RemoveContainer" containerID="4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd"
Oct 30 21:34:20.680827 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.680805    4852 scope.go:122] "RemoveContainer" containerID="4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd"
Oct 30 21:34:20.681129 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:34:20.681107    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd\": container with ID starting with 4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd not found: ID does not exist" containerID="4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd"
Oct 30 21:34:20.681203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.681141    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd"} err="failed to get container status \"4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd\": rpc error: code = NotFound desc = could not find container \"4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd\": container with ID starting with 4c6370d8d859f4f32cc54e104ada7e5926205554901242fa1cd85b8a93df81bd not found: ID does not exist"
Oct 30 21:34:20.694044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.694013    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"]
Oct 30 21:34:20.698205 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:20.698182    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-317a33/premium-w1-317a33-kserve-5b7f75fd95-ms7x4"]
Oct 30 21:34:21.557757 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:34:21.557723    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:34:22.003482 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:22.003409    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" path="/var/lib/kubelet/pods/775b7bb3-76c8-41fd-876b-f3646c25b77b/volumes"
Oct 30 21:34:25.718249 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:25.718207    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:25.718758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:25.718224    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" podUID="979b5e79-af17-48e2-969a-96a6ef1b4a37" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:34:26.471597 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.471562    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"]
Oct 30 21:34:26.471835 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.471788    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r" podUID="cbc65f10-d54b-4238-82ff-71b4a623b7ca" containerName="maas-api" containerID="cri-o://562ca65dbffa2a91f5b77ac0985dccf0103b7463024fad7a3868a204a383ce77" gracePeriod=30
Oct 30 21:34:26.518886 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.518856    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.668685 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.668648    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.709571 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.709542    4852 generic.go:356] "Generic (PLEG): container finished" podID="cbc65f10-d54b-4238-82ff-71b4a623b7ca" containerID="562ca65dbffa2a91f5b77ac0985dccf0103b7463024fad7a3868a204a383ce77" exitCode=0
Oct 30 21:34:26.709707 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.709612    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r" event={"ID":"cbc65f10-d54b-4238-82ff-71b4a623b7ca","Type":"ContainerDied","Data":"562ca65dbffa2a91f5b77ac0985dccf0103b7463024fad7a3868a204a383ce77"}
Oct 30 21:34:26.726962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.726903    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 21:34:26.757634 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.757579    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.836014 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.835983    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cbc65f10-d54b-4238-82ff-71b4a623b7ca-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc65f10-d54b-4238-82ff-71b4a623b7ca-openshift-service-ca\") pod \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") "
Oct 30 21:34:26.836203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.836032    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-maas-api-tls\") pod \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") "
Oct 30 21:34:26.836203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.836052    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-metrics-certs\") pod \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") "
Oct 30 21:34:26.836203 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.836122    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbc65f10-d54b-4238-82ff-71b4a623b7ca-kube-api-access-fnfn8\" (UniqueName: \"kubernetes.io/projected/cbc65f10-d54b-4238-82ff-71b4a623b7ca-kube-api-access-fnfn8\") pod \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\" (UID: \"cbc65f10-d54b-4238-82ff-71b4a623b7ca\") "
Oct 30 21:34:26.836627 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.836595    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbc65f10-d54b-4238-82ff-71b4a623b7ca-openshift-service-ca" pod "cbc65f10-d54b-4238-82ff-71b4a623b7ca" (UID: "cbc65f10-d54b-4238-82ff-71b4a623b7ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:26.838342 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.838310    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc65f10-d54b-4238-82ff-71b4a623b7ca-kube-api-access-fnfn8" pod "cbc65f10-d54b-4238-82ff-71b4a623b7ca" (UID: "cbc65f10-d54b-4238-82ff-71b4a623b7ca"). InnerVolumeSpecName "kube-api-access-fnfn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:26.838488 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.838398    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-maas-api-tls" pod "cbc65f10-d54b-4238-82ff-71b4a623b7ca" (UID: "cbc65f10-d54b-4238-82ff-71b4a623b7ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:26.838537 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.838499    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-metrics-certs" pod "cbc65f10-d54b-4238-82ff-71b4a623b7ca" (UID: "cbc65f10-d54b-4238-82ff-71b4a623b7ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:34:26.937678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.937634    4852 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc65f10-d54b-4238-82ff-71b4a623b7ca-openshift-service-ca\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:26.937678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.937671    4852 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-maas-api-tls\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:26.937678 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.937684    4852 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc65f10-d54b-4238-82ff-71b4a623b7ca-metrics-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:26.937935 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:26.937697    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnfn8\" (UniqueName: \"kubernetes.io/projected/cbc65f10-d54b-4238-82ff-71b4a623b7ca-kube-api-access-fnfn8\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:27.220893 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:27.220858    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:27.715995 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:27.715964    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"
Oct 30 21:34:27.716208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:27.715962    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r" event={"ID":"cbc65f10-d54b-4238-82ff-71b4a623b7ca","Type":"ContainerDied","Data":"6caaac5a24a72b407ff6029407708ca7305d00ab1b3e7227db40544d7cde14a5"}
Oct 30 21:34:27.716208 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:27.716105    4852 scope.go:122] "RemoveContainer" containerID="562ca65dbffa2a91f5b77ac0985dccf0103b7463024fad7a3868a204a383ce77"
Oct 30 21:34:27.742463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:27.742434    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"]
Oct 30 21:34:27.750135 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:27.750108    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-317a33-57f4765696-wgg8r"]
Oct 30 21:34:28.003890 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:28.003798    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cbc65f10-d54b-4238-82ff-71b4a623b7ca" path="/var/lib/kubelet/pods/cbc65f10-d54b-4238-82ff-71b4a623b7ca/volumes"
Oct 30 21:34:33.159329 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:33.159282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:33.232170 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:33.232132    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:33.538206 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:33.538170    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"]
Oct 30 21:34:33.538429 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:33.538405    4852 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy" containerID="cri-o://77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149" gracePeriod=30
Oct 30 21:34:33.584151 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:33.584114    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:34.955209 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:34.955162    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-dtgm8_45994274-c5d2-4041-8f70-7dc2304f1580/manager/0.log"
Oct 30 21:34:35.158044 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.158001    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-dpr68_cc0ffb68-6f72-4b62-8069-da67c20f530a/manager/0.log"
Oct 30 21:34:35.705938 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.705913    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 21:34:35.715061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.715035    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/979b5e79-af17-48e2-969a-96a6ef1b4a37-kube-api-access-t972h\" (UniqueName: \"kubernetes.io/projected/979b5e79-af17-48e2-969a-96a6ef1b4a37-kube-api-access-t972h\") pod \"979b5e79-af17-48e2-969a-96a6ef1b4a37\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") "
Oct 30 21:34:35.715213 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.715079    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/979b5e79-af17-48e2-969a-96a6ef1b4a37-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/979b5e79-af17-48e2-969a-96a6ef1b4a37-plugins-config-volume\") pod \"979b5e79-af17-48e2-969a-96a6ef1b4a37\" (UID: \"979b5e79-af17-48e2-969a-96a6ef1b4a37\") "
Oct 30 21:34:35.715490 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.715468    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/979b5e79-af17-48e2-969a-96a6ef1b4a37-plugins-config-volume" pod "979b5e79-af17-48e2-969a-96a6ef1b4a37" (UID: "979b5e79-af17-48e2-969a-96a6ef1b4a37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:35.717214 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.717191    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/979b5e79-af17-48e2-969a-96a6ef1b4a37-kube-api-access-t972h" pod "979b5e79-af17-48e2-969a-96a6ef1b4a37" (UID: "979b5e79-af17-48e2-969a-96a6ef1b4a37"). InnerVolumeSpecName "kube-api-access-t972h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:35.733577 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.733552    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 21:34:35.761260 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.761222    4852 generic.go:356] "Generic (PLEG): container finished" podID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerID="25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a" exitCode=0
Oct 30 21:34:35.761463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.761263    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"
Oct 30 21:34:35.761463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.761301    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" event={"ID":"d67b319c-7082-40df-b5ef-b3eb6b43ad90","Type":"ContainerDied","Data":"25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a"}
Oct 30 21:34:35.761463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.761358    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" event={"ID":"d67b319c-7082-40df-b5ef-b3eb6b43ad90","Type":"ContainerDied","Data":"91d3ec5162441e98698f511611aff2163e76ff876122053093062945652f974d"}
Oct 30 21:34:35.761463 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.761404    4852 scope.go:122] "RemoveContainer" containerID="25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a"
Oct 30 21:34:35.762972 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.762946    4852 generic.go:356] "Generic (PLEG): container finished" podID="979b5e79-af17-48e2-969a-96a6ef1b4a37" containerID="7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964" exitCode=0
Oct 30 21:34:35.763103 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.763040    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" event={"ID":"979b5e79-af17-48e2-969a-96a6ef1b4a37","Type":"ContainerDied","Data":"7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964"}
Oct 30 21:34:35.763162 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.763104    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz" event={"ID":"979b5e79-af17-48e2-969a-96a6ef1b4a37","Type":"ContainerDied","Data":"2161187690efb75ed776e747b2caaec54310acaab541792ff4617a3e1181024e"}
Oct 30 21:34:35.763162 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.763046    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"
Oct 30 21:34:35.772016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.771998    4852 scope.go:122] "RemoveContainer" containerID="25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a"
Oct 30 21:34:35.772324 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:34:35.772305    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a\": container with ID starting with 25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a not found: ID does not exist" containerID="25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a"
Oct 30 21:34:35.772416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.772331    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a"} err="failed to get container status \"25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a\": rpc error: code = NotFound desc = could not find container \"25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a\": container with ID starting with 25f291d9ae7e076f40cb87bbe9b53ee4c798f96742db3eef3ab22fa6e06be71a not found: ID does not exist"
Oct 30 21:34:35.772416 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.772346    4852 scope.go:122] "RemoveContainer" containerID="7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964"
Oct 30 21:34:35.781801 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.781784    4852 scope.go:122] "RemoveContainer" containerID="7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964"
Oct 30 21:34:35.782064 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:34:35.782041    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964\": container with ID starting with 7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964 not found: ID does not exist" containerID="7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964"
Oct 30 21:34:35.782130 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.782070    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964"} err="failed to get container status \"7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964\": rpc error: code = NotFound desc = could not find container \"7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964\": container with ID starting with 7440c2b99ee6a403a123493d6fa08899e72fa2240743c107d85e93299a038964 not found: ID does not exist"
Oct 30 21:34:35.789292 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.789258    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"]
Oct 30 21:34:35.795419 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.795376    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-317a33-74fbf67dcc-4tdfz"]
Oct 30 21:34:35.816426 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.816399    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d67b319c-7082-40df-b5ef-b3eb6b43ad90-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b319c-7082-40df-b5ef-b3eb6b43ad90-plugins-config-volume\") pod \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") "
Oct 30 21:34:35.816551 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.816489    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d67b319c-7082-40df-b5ef-b3eb6b43ad90-kube-api-access-kn8xw\" (UniqueName: \"kubernetes.io/projected/d67b319c-7082-40df-b5ef-b3eb6b43ad90-kube-api-access-kn8xw\") pod \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\" (UID: \"d67b319c-7082-40df-b5ef-b3eb6b43ad90\") "
Oct 30 21:34:35.816767 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.816753    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t972h\" (UniqueName: \"kubernetes.io/projected/979b5e79-af17-48e2-969a-96a6ef1b4a37-kube-api-access-t972h\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:35.816829 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.816772    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/979b5e79-af17-48e2-969a-96a6ef1b4a37-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:35.816868 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.816845    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d67b319c-7082-40df-b5ef-b3eb6b43ad90-plugins-config-volume" pod "d67b319c-7082-40df-b5ef-b3eb6b43ad90" (UID: "d67b319c-7082-40df-b5ef-b3eb6b43ad90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:35.818747 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.818722    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d67b319c-7082-40df-b5ef-b3eb6b43ad90-kube-api-access-kn8xw" pod "d67b319c-7082-40df-b5ef-b3eb6b43ad90" (UID: "d67b319c-7082-40df-b5ef-b3eb6b43ad90"). InnerVolumeSpecName "kube-api-access-kn8xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:35.922322 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.922283    4852 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d67b319c-7082-40df-b5ef-b3eb6b43ad90-plugins-config-volume\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:35.922535 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:35.922315    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn8xw\" (UniqueName: \"kubernetes.io/projected/d67b319c-7082-40df-b5ef-b3eb6b43ad90-kube-api-access-kn8xw\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:36.003297 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:36.003219    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="979b5e79-af17-48e2-969a-96a6ef1b4a37" path="/var/lib/kubelet/pods/979b5e79-af17-48e2-969a-96a6ef1b4a37/volumes"
Oct 30 21:34:36.010807 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:36.010778    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-dwfh7_43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4/manager/0.log"
Oct 30 21:34:36.080614 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:36.080582    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"]
Oct 30 21:34:36.084779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:36.084754    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt"]
Oct 30 21:34:36.718382 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:36.718285    4852 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-317a33-849fd968c5-76klt" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.54:50052\" within 1s: context deadline exceeded"
Oct 30 21:34:36.718571 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:34:36.718453    4852 logging.go:55] [core] [Channel #11115 SubChannel #11116]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.54:50052", ServerName: "10.133.0.54:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.54:50052: operation was canceled"
Oct 30 21:34:37.167102 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:37.167058    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:34:38.003051 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.003016    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" path="/var/lib/kubelet/pods/d67b319c-7082-40df-b5ef-b3eb6b43ad90/volumes"
Oct 30 21:34:38.784339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.784306    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 21:34:38.784758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.784699    4852 generic.go:356] "Generic (PLEG): container finished" podID="f5365800-223d-4943-8936-0fbf3b641703" containerID="77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149" exitCode=0
Oct 30 21:34:38.784758 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.784749    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" event={"ID":"f5365800-223d-4943-8936-0fbf3b641703","Type":"ContainerDied","Data":"77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149"}
Oct 30 21:34:38.784897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.784802    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt" event={"ID":"f5365800-223d-4943-8936-0fbf3b641703","Type":"ContainerDied","Data":"d21d818ef308e34c9c24925cd12a59a24555c8773d5f4b0294f991a55ede2bd5"}
Oct 30 21:34:38.784897 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.784834    4852 scope.go:122] "RemoveContainer" containerID="77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149"
Oct 30 21:34:38.793967 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.793946    4852 scope.go:122] "RemoveContainer" containerID="77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149"
Oct 30 21:34:38.794228 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:34:38.794208    4852 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149\": container with ID starting with 77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149 not found: ID does not exist" containerID="77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149"
Oct 30 21:34:38.794275 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.794236    4852 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149"} err="failed to get container status \"77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149\": rpc error: code = NotFound desc = could not find container \"77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149\": container with ID starting with 77d4e26298a98473da8d76f24a9fb83e851054140a484e52a588ebca4e658149 not found: ID does not exist"
Oct 30 21:34:38.846564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846493    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-certs\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.846564 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846536    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-kube-api-access-qb76p\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-kube-api-access-qb76p\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.846769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846602    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5365800-223d-4943-8936-0fbf3b641703-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5365800-223d-4943-8936-0fbf3b641703-istiod-ca-cert\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.846769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846658    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-credential-socket\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.846769 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846702    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-envoy\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.846941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846788    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-certs" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:38.846941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846834    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f5365800-223d-4943-8936-0fbf3b641703-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5365800-223d-4943-8936-0fbf3b641703-istio-podinfo\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.846941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846891    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-data\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.846941 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846932    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-istio-token\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-istio-token\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.847173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846953    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-credential-socket" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:38.847173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.846960    4852 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-socket\") pod \"f5365800-223d-4943-8936-0fbf3b641703\" (UID: \"f5365800-223d-4943-8936-0fbf3b641703\") "
Oct 30 21:34:38.847173 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847064    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5365800-223d-4943-8936-0fbf3b641703-istiod-ca-cert" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:38.847346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847222    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-socket" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:38.847346 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847239    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-data" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:38.847479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847374    4852 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5365800-223d-4943-8936-0fbf3b641703-istiod-ca-cert\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.847479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847419    4852 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-credential-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.847479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847429    4852 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-data\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.847479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847437    4852 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-socket\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.847479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.847450    4852 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-workload-certs\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.849181 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.849152    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-istio-token" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:38.849305 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.849290    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-envoy" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:38.849381 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.849299    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5365800-223d-4943-8936-0fbf3b641703-istio-podinfo" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:34:38.849466 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.849434    4852 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-kube-api-access-qb76p" pod "f5365800-223d-4943-8936-0fbf3b641703" (UID: "f5365800-223d-4943-8936-0fbf3b641703"). InnerVolumeSpecName "kube-api-access-qb76p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:38.948210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.948178    4852 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-istio-token\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.948210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.948206    4852 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb76p\" (UniqueName: \"kubernetes.io/projected/f5365800-223d-4943-8936-0fbf3b641703-kube-api-access-qb76p\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.948210 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.948216    4852 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5365800-223d-4943-8936-0fbf3b641703-istio-envoy\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:38.948479 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:38.948225    4852 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5365800-223d-4943-8936-0fbf3b641703-istio-podinfo\") on node \"ip-10-0-135-178.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:39.791040 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:39.791009    4852 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"
Oct 30 21:34:39.814317 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:39.814284    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"]
Oct 30 21:34:39.818373 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:39.818344    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-317a33-openshift-default-6d6f98675-bvvrt"]
Oct 30 21:34:40.003630 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:40.003591    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5365800-223d-4943-8936-0fbf3b641703" path="/var/lib/kubelet/pods/f5365800-223d-4943-8936-0fbf3b641703/volumes"
Oct 30 21:34:41.407763 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:41.407713    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:41.539596 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:41.538035    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:41.635998 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:41.635962    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:41.737348 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:41.737233    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:42.819319 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:42.819282    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:44.619370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:44.619321    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:44.720800 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:44.720769    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:48.714218 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:48.714180    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:50.016994 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:50.016959    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:34:50.646698 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:50.646662    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:51.557164 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:34:51.557129    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:34:52.775791 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:52.775764    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-5fe17c55-hp2jn_6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e/revoke-keys/0.log"
Oct 30 21:34:53.499770 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:53.499736    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-b8673e82-cg8td_e5bd3453-1c90-4cb4-b648-5484b42fa217/revoke-keys/0.log"
Oct 30 21:34:56.078903 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:56.078867    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-dtgm8_45994274-c5d2-4041-8f70-7dc2304f1580/manager/0.log"
Oct 30 21:34:56.257943 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:56.257910    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-dpr68_cc0ffb68-6f72-4b62-8069-da67c20f530a/manager/0.log"
Oct 30 21:34:57.132674 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:57.132642    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-dwfh7_43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4/manager/0.log"
Oct 30 21:34:57.891174 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:57.891143    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57dc67465-62gx5_0054779a-9980-4544-a064-233ea0e46755/istio-proxy/0.log"
Oct 30 21:34:58.081155 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:58.081126    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-flf8w_e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9/discovery/0.log"
Oct 30 21:34:58.802870 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:58.802839    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77f556d647-b8h86_d2ea773e-9dd0-4894-8610-4c5c8f465087/istio-proxy/0.log"
Oct 30 21:34:59.742767 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:34:59.742737    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-689986b98c-m284n_da498268-ef23-4b1d-812b-e88573f2268e/istio-proxy/0.log"
Oct 30 21:35:01.179926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:01.179888    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-hl2cc_a0cd0d78-ca07-499a-b928-3ce2b1d00627/main/0.log"
Oct 30 21:35:01.541153 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:01.541122    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-x27k5_3f7f9d76-006d-4453-a291-4037dd19730d/main/0.log"
Oct 30 21:35:01.723417 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:01.723359    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8gw4gd_bc074dfb-fc56-4d06-aec4-ba0da629998e/main/0.log"
Oct 30 21:35:02.240610 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:02.240576    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-z94z5_06b775dd-29f6-409e-8725-a476c704e31c/main/0.log"
Oct 30 21:35:05.980231 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:05.980198    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:12.938033 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:12.937996    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:13.030023 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:13.029978    4852 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:13.030604 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:13.030581    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"]
Oct 30 21:35:13.034364 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:13.034340    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8673e82-cg8td"]
Oct 30 21:35:14.002117 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:14.002086    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5bd3453-1c90-4cb4-b648-5484b42fa217" path="/var/lib/kubelet/pods/e5bd3453-1c90-4cb4-b648-5484b42fa217/volumes"
Oct 30 21:35:17.593362 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:17.593329    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tck7z_754d8def-099b-4fdb-859c-0f318bf5fd38/global-pull-secret-syncer/0.log"
Oct 30 21:35:17.846796 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:17.846720    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bsg94_fd3db454-59fb-46b2-90e9-3b7f4c11d5bb/konnectivity-agent/0.log"
Oct 30 21:35:18.089049 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:18.089020    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-178.us-east-2.compute.internal_3d0d587662eb50ad2820bb8aa4c2c1ec/haproxy/0.log"
Oct 30 21:35:21.557023 ip-10-0-135-178 kubenswrapper[4852]: E1030 21:35:21.556981    4852 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-rjlsv" containerName="extractor"
Oct 30 21:35:25.022789 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:25.022741    4852 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"]
Oct 30 21:35:25.025720 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:25.025692    4852 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5fe17c55-hp2jn"]
Oct 30 21:35:26.003367 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:26.003326    4852 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e" path="/var/lib/kubelet/pods/6c1d1d8f-fbec-42a6-ac66-4fa53c72ec7e/volumes"
Oct 30 21:35:26.904376 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:26.904347    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-dtgm8_45994274-c5d2-4041-8f70-7dc2304f1580/manager/0.log"
Oct 30 21:35:26.940957 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:26.940929    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-dpr68_cc0ffb68-6f72-4b62-8069-da67c20f530a/manager/0.log"
Oct 30 21:35:27.614273 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:27.614237    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-dwfh7_43f6af0c-cf27-4018-ba25-fbbdbf7f7eb4/manager/0.log"
Oct 30 21:35:29.671981 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.671950    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qfl7v_70f4b787-e9c9-4e85-894d-27d370f8766d/node-exporter/0.log"
Oct 30 21:35:29.722926 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.722895    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qfl7v_70f4b787-e9c9-4e85-894d-27d370f8766d/kube-rbac-proxy/0.log"
Oct 30 21:35:29.748061 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.748035    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qfl7v_70f4b787-e9c9-4e85-894d-27d370f8766d/init-textfile/0.log"
Oct 30 21:35:29.864706 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.864667    4852 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"]
Oct 30 21:35:29.865087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865060    4852 cpu_manager.go:414] "containerMap: removing container" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main"
Oct 30 21:35:29.865087 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865081    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865096    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865104    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865118    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865125    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865138    4852 cpu_manager.go:414] "containerMap: removing container" podUID="19bd7c20-fc68-4215-9b00-5ef1629d3a5c" containerName="revoke-keys"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865145    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19bd7c20-fc68-4215-9b00-5ef1629d3a5c" containerName="revoke-keys"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865160    4852 cpu_manager.go:414] "containerMap: removing container" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865167    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865193    4852 cpu_manager.go:414] "containerMap: removing container" podUID="979b5e79-af17-48e2-969a-96a6ef1b4a37" containerName="payload-processing"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865200    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="979b5e79-af17-48e2-969a-96a6ef1b4a37" containerName="payload-processing"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865222    4852 cpu_manager.go:414] "containerMap: removing container" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865229    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865239    4852 cpu_manager.go:414] "containerMap: removing container" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865246    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865255    4852 cpu_manager.go:414] "containerMap: removing container" podUID="cbc65f10-d54b-4238-82ff-71b4a623b7ca" containerName="maas-api"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865263    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbc65f10-d54b-4238-82ff-71b4a623b7ca" containerName="maas-api"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865274    4852 cpu_manager.go:414] "containerMap: removing container" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerName="payload-pre-processing"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865281    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerName="payload-pre-processing"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865294    4852 cpu_manager.go:414] "containerMap: removing container" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865301    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f921d6df-789f-4dca-8a1e-14c3d443fbc7" containerName="cleanup"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865318    4852 cpu_manager.go:414] "containerMap: removing container" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy"
Oct 30 21:35:29.865337 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865325    4852 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47aaf1c6-f339-46d3-b3d5-a65397293537" containerName="istio-proxy"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865444    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="979b5e79-af17-48e2-969a-96a6ef1b4a37" containerName="payload-processing"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865461    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="60f52450-d4d3-4903-8be4-c97eac38e4f9" containerName="istio-proxy"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865471    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbc65f10-d54b-4238-82ff-71b4a623b7ca" containerName="maas-api"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865479    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="d67b319c-7082-40df-b5ef-b3eb6b43ad90" containerName="payload-pre-processing"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865492    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="0066ff78-850a-4603-8eaa-0b533e03f28d" containerName="cleanup"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865505    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="775b7bb3-76c8-41fd-876b-f3646c25b77b" containerName="main"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865515    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="cec79507-2620-4011-9ee9-04547f6810de" containerName="cleanup"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865525    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5365800-223d-4943-8936-0fbf3b641703" containerName="istio-proxy"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865534    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="19bd7c20-fc68-4215-9b00-5ef1629d3a5c" containerName="revoke-keys"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865541    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3b136f0-4b7e-4a66-bb5f-e6f296a7c19d" containerName="istio-proxy"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865552    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="71a8ec2e-6164-49dd-98d4-a23312d456f2" containerName="istio-proxy"
Oct 30 21:35:29.866585 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.865562    4852 memory_manager.go:365] "RemoveStaleState removing state" podUID="40028a96-aedb-4c5a-9651-f1784ba3f52f" containerName="cleanup"
Oct 30 21:35:29.869114 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.869092    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:29.876989 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.876966    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8ln49\"/\"openshift-service-ca.crt\""
Oct 30 21:35:29.877137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.877056    4852 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8ln49\"/\"kube-root-ca.crt\""
Oct 30 21:35:29.877137 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.877117    4852 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8ln49\"/\"default-dockercfg-n87st\""
Oct 30 21:35:29.879245 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.879216    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"]
Oct 30 21:35:29.928603 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.928511    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-lib-modules\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:29.928779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.928603    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-proc\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:29.928779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.928643    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-podres\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:29.928779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.928668    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f74gc\" (UniqueName: \"kubernetes.io/projected/a316fe9e-4828-4741-a268-bd3424e1226e-kube-api-access-f74gc\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:29.928779 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:29.928766    4852 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-sys\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.025015 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.024980    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6ec22351-7ca8-448c-b9e1-fb43f8ec6257/prometheus/0.log"
Oct 30 21:35:30.029219 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029196    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-sys\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029241    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-lib-modules\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029281    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-proc\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029308    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-podres\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029339 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029313    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-sys\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029337    4852 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f74gc\" (UniqueName: \"kubernetes.io/projected/a316fe9e-4828-4741-a268-bd3424e1226e-kube-api-access-f74gc\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029375    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-proc\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029418    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-podres\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.029528 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.029433    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a316fe9e-4828-4741-a268-bd3424e1226e-lib-modules\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.037809 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.037774    4852 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f74gc\" (UniqueName: \"kubernetes.io/projected/a316fe9e-4828-4741-a268-bd3424e1226e-kube-api-access-f74gc\") pod \"perf-node-gather-daemonset-jrcrr\" (UID: \"a316fe9e-4828-4741-a268-bd3424e1226e\") " pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.052600 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.052573    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6ec22351-7ca8-448c-b9e1-fb43f8ec6257/config-reloader/0.log"
Oct 30 21:35:30.079089 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.079061    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6ec22351-7ca8-448c-b9e1-fb43f8ec6257/thanos-sidecar/0.log"
Oct 30 21:35:30.107016 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.106984    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6ec22351-7ca8-448c-b9e1-fb43f8ec6257/kube-rbac-proxy-web/0.log"
Oct 30 21:35:30.136100 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.136072    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6ec22351-7ca8-448c-b9e1-fb43f8ec6257/kube-rbac-proxy/0.log"
Oct 30 21:35:30.171076 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.171032    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6ec22351-7ca8-448c-b9e1-fb43f8ec6257/kube-rbac-proxy-thanos/0.log"
Oct 30 21:35:30.182624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.182550    4852 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:30.198304 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.198278    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6ec22351-7ca8-448c-b9e1-fb43f8ec6257/init-config-reloader/0.log"
Oct 30 21:35:30.456787 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:30.456760    4852 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"]
Oct 30 21:35:30.459109 ip-10-0-135-178 kubenswrapper[4852]: W1030 21:35:30.459081    4852 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda316fe9e_4828_4741_a268_bd3424e1226e.slice/crio-c9af663d30d6ef3b970e3c8b7d886356ab5bc543d87da174d4bfa3ea8a8a8d58 WatchSource:0}: Error finding container c9af663d30d6ef3b970e3c8b7d886356ab5bc543d87da174d4bfa3ea8a8a8d58: Status 404 returned error can't find the container with id c9af663d30d6ef3b970e3c8b7d886356ab5bc543d87da174d4bfa3ea8a8a8d58
Oct 30 21:35:31.072987 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:31.072953    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr" event={"ID":"a316fe9e-4828-4741-a268-bd3424e1226e","Type":"ContainerStarted","Data":"e06e7d2ec8997fd2237904e7f5da94e8d13857ba434cc921fa72f95712af6539"}
Oct 30 21:35:31.073438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:31.073014    4852 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr" event={"ID":"a316fe9e-4828-4741-a268-bd3424e1226e","Type":"ContainerStarted","Data":"c9af663d30d6ef3b970e3c8b7d886356ab5bc543d87da174d4bfa3ea8a8a8d58"}
Oct 30 21:35:31.073438 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:31.073040    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:31.089106 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:31.089036    4852 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr" podStartSLOduration=2.089020717 podStartE2EDuration="2.089020717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:35:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:31.088355015 +0000 UTC m=+3439.203089206" watchObservedRunningTime="2026-10-30 21:35:31.089020717 +0000 UTC m=+3439.203754908"
Oct 30 21:35:34.054129 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:34.054083    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:35:34.095624 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:34.095585    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:35:36.605970 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:36.605936    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ck9ws_14627328-5fa1-4c44-93a1-94b03f420bc7/dns/0.log"
Oct 30 21:35:36.633474 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:36.633438    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ck9ws_14627328-5fa1-4c44-93a1-94b03f420bc7/kube-rbac-proxy/0.log"
Oct 30 21:35:36.868718 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:36.868627    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g9hwf_9c7b0506-8e4a-43bd-b7a6-e58e9879d893/dns-node-resolver/0.log"
Oct 30 21:35:37.089990 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:37.089960    4852 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8ln49/perf-node-gather-daemonset-jrcrr"
Oct 30 21:35:37.900083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:37.900056    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-29897_4f7516cb-f8ee-4e50-88ae-f2cd6cd40087/node-ca/0.log"
Oct 30 21:35:39.125258 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:39.125226    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:35:39.236083 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:39.236042    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:35:40.140188 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:40.140152    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57dc67465-62gx5_0054779a-9980-4544-a064-233ea0e46755/istio-proxy/0.log"
Oct 30 21:35:42.694948 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:42.694857    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-flf8w_e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9/discovery/0.log"
Oct 30 21:35:43.368460 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:43.368430    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77f556d647-b8h86_d2ea773e-9dd0-4894-8610-4c5c8f465087/istio-proxy/0.log"
Oct 30 21:35:43.563335 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:43.563305    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-689986b98c-m284n_da498268-ef23-4b1d-812b-e88573f2268e/istio-proxy/0.log"
Oct 30 21:35:44.258071 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:44.258039    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:35:44.363735 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:44.363701    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-57dc67465-62gx5_0054779a-9980-4544-a064-233ea0e46755/istio-proxy/0.log"
Oct 30 21:35:44.376887 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:44.376862    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-jclms_f92a4844-d433-481b-a4cf-771108c24109/manager/0.log"
Oct 30 21:35:46.462012 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:46.461971    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-flf8w_e5f8904a-5940-4a70-a5cd-6dc0ebf03bb9/discovery/0.log"
Oct 30 21:35:46.694962 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:46.694927    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-77f556d647-b8h86_d2ea773e-9dd0-4894-8610-4c5c8f465087/istio-proxy/0.log"
Oct 30 21:35:46.839795 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:46.839758    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-689986b98c-m284n_da498268-ef23-4b1d-812b-e88573f2268e/istio-proxy/0.log"
Oct 30 21:35:46.911309 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:46.911279    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mrq2q_f314cf90-719e-4d6d-8745-0ff46bbbf9f8/serve-healthcheck-canary/0.log"
Oct 30 21:35:48.114760 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:48.114728    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rjlsv_5108533b-2d1b-488f-a03c-fd5beb4f34f5/kube-rbac-proxy/0.log"
Oct 30 21:35:48.140689 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:48.140659    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rjlsv_5108533b-2d1b-488f-a03c-fd5beb4f34f5/exporter/0.log"
Oct 30 21:35:48.167370 ip-10-0-135-178 kubenswrapper[4852]: I1030 21:35:48.167342    4852 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rjlsv_5108533b-2d1b-488f-a03c-fd5beb4f34f5/extractor/0.log"